Aug 04 15:56:10.541054 ip-10-0-132-227 systemd[1]: Starting Kubernetes Kubelet... Aug 04 15:56:10.992995 ip-10-0-132-227 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 15:56:10.992995 ip-10-0-132-227 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 15:56:10.992995 ip-10-0-132-227 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 15:56:10.992995 ip-10-0-132-227 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 15:56:10.992995 ip-10-0-132-227 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 15:56:10.994790 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:10.994680 2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 15:56:11.000589 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000570 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:56:11.000589 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000590 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000593 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000597 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000600 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000603 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000606 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000609 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000612 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000614 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000617 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000620 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000623 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000626 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000632 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000635 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000637 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000640 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000642 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000645 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:56:11.000656 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000648 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000650 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000653 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000656 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000659 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000661 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000664 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000667 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000670 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000672 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000675 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000678 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000680 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000683 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000685 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000689 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000691 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000694 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000697 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:56:11.001124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000701 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000705 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000708 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000710 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000713 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000715 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000718 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000730 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000733 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000735 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000738 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000740 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000743 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000745 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000747 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000751 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000754 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000756 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000759 2556 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000762 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:56:11.001583 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000765 2556 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000767 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000770 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000772 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000775 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000777 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000780 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000782 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000785 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000787 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000790 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000794 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000798 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000802 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000805 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000807 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000811 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000813 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000817 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:56:11.002124 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000820 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000824 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000827 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000829 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000832 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000834 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000837 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000839 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000843 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000846 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000848 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000851 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000854 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000856 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000859 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000862 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000864 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000867 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000869 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000872 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:56:11.002572 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.000874 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001614 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001623 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001628 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001631 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001634 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001637 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001639 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001642 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001644 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001648 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001651 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001653 2556 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001656 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001659 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001661 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001664 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001666 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001669 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001671 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:56:11.003074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001675 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001678 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001681 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001683 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001686 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001689 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001691 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001695 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001699 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001702 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001704 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001707 2556 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001709 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001712 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001714 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001717 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001720 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001736 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001738 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:56:11.003567 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001741 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001744 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001746 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001751 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001755 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001758 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001760 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001763 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001765 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001768 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001770 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001773 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001775 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001779 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001782 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001784 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001787 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001790 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001792 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001795 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:56:11.004040 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001797 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001800 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001803 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001805 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001807 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001810 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001812 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001815 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001817 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001820 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001822 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001824 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001827 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001829 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001832 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001834 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001837 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001841 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001844 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001846 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:56:11.004541 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001849 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001852 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001854 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001857 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001859 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001862 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001864 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001867 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001870 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001872 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001875 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001877 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001879 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001882 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001884 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001887 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001889 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001892 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001894 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:56:11.005074 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001897 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.001899 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002308 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002314 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002317 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002320 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002322 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002325 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002327 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002330 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002333 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002336 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002338 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002341 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002344 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002347 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002350 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002352 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002355 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002357 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:56:11.005544 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002360 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002364 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002368 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002371 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002374 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002377 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002379 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002382 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002385 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002388 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002391 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002394 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002396 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002399 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002402 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002405 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002408 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002412 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002415 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:56:11.006049 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002419 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002422 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002425 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002427 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002431 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002433 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002437 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002440 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002443 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002445 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002448 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002451 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002453 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002456 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002459 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002462 2556 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002464 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002467 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002469 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002472 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:56:11.006513 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002474 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002476 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002479 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002482 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002484 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002486 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002489 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002492 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002494 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002497 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002499 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002502 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002504 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002507 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002509 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002511 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002517 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002520 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002523 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:56:11.007041 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002525 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002528 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002530 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002533 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002535 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002538 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002540 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002543 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002545 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002547 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002550 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002552 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002555 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002557 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002560 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002562 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002565 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002567 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002569 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002572 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:56:11.007498 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002575 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002578 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.002580 2556 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002655 2556 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002671 2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002678 2556 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002682 2556 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002687 2556 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002690 2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002695 2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002699 2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002704 2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002707 2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002710 2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002713 2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002716 2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002719 2556 flags.go:64] FLAG: --cgroup-root="" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002735 2556 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002739 2556 flags.go:64] FLAG: --client-ca-file="" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002741 2556 flags.go:64] FLAG: --cloud-provider="external" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002744 2556 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002749 2556 flags.go:64] FLAG: --cluster-domain="" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002752 2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002755 2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 15:56:11.007985 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002758 2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002762 2556 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002769 2556 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002772 2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002775 2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002778 2556 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002782 2556 flags.go:64] FLAG: --contention-profiling="false" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002785 2556 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002788 2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002791 2556 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002794 2556 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002798 2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002801 2556 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002804 2556 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002807 2556 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002810 2556 flags.go:64] FLAG: --enable-server="true" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002812 2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002819 2556 flags.go:64] FLAG: --event-burst="100" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002822 2556 flags.go:64] FLAG: --event-qps="50" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002825 2556 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002833 2556 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002837 2556 flags.go:64] FLAG: --eviction-hard="" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002841 2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002844 2556 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002847 2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 15:56:11.008549 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002850 2556 flags.go:64] FLAG: --eviction-soft="" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002852 2556 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002855 2556 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002858 2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002861 2556 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002864 2556 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002866 2556 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002869 2556 flags.go:64] FLAG: --feature-gates="" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002873 2556 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002878 2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002881 2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002884 2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002887 2556 flags.go:64] FLAG: --healthz-port="10248" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002890 2556 flags.go:64] FLAG: --help="false" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002893 2556 flags.go:64] FLAG: --hostname-override="ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002896 2556 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002899 2556 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002902 2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002906 2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002909 2556 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002912 2556 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002915 2556 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002918 2556 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 15:56:11.009167 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002920 2556 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002925 2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002928 2556 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002931 2556 flags.go:64] FLAG: --kube-reserved="" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002934 2556 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002937 2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002940 2556 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002943 2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002945 2556 flags.go:64] FLAG: --lock-file="" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002948 2556 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002951 2556 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002954 2556 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002959 2556 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002962 2556 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002965 2556 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002967 2556 flags.go:64] FLAG: --logging-format="text" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002970 2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002973 2556 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002977 2556 flags.go:64] FLAG: --manifest-url="" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002980 2556 flags.go:64] FLAG: --manifest-url-header="" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002985 2556 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002987 2556 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002991 2556 flags.go:64] FLAG: --max-pods="110" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002994 2556 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.002997 2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 15:56:11.009763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003001 2556 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003004 2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003007 2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003010 2556 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003012 2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003020 2556 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003022 2556 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003025 2556 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003030 2556 flags.go:64] FLAG: --pod-cidr="" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003033 2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003038 2556 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003041 2556 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003044 2556 flags.go:64] FLAG: --pods-per-core="0" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003047 2556 flags.go:64] FLAG: --port="10250" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003050 2556 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003053 2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06e346296ae3b52cb" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003057 2556 flags.go:64] FLAG: --qos-reserved="" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003060 2556 flags.go:64] FLAG: --read-only-port="10255" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003063 2556 flags.go:64] FLAG: --register-node="true" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003066 2556 flags.go:64] FLAG: --register-with-taints="" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003069 2556 flags.go:64] FLAG: --registry-burst="10" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003072 2556 flags.go:64] FLAG: --registry-qps="5" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003075 2556 flags.go:64] FLAG: --reserved-cpus="" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003078 2556 flags.go:64] FLAG: --reserved-memory="" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003082 2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 15:56:11.010432 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003086 2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003089 2556 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003092 2556 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003095 2556 flags.go:64] FLAG: --runonce="false" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003098 2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003101 2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003104 2556 flags.go:64] FLAG: --seccomp-default="false" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003107 2556 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003110 2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003113 2556 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003116 2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003119 2556 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003122 2556 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003125 2556 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003128 2556 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003131 2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003136 2556 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003139 2556 flags.go:64] FLAG: --system-cgroups="" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003142 2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003147 2556 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003150 2556 flags.go:64] FLAG: --tls-cert-file="" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003153 2556 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003157 2556 flags.go:64] FLAG: --tls-min-version="" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003160 2556 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003163 2556 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 15:56:11.011066 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003166 2556 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003169 2556 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003172 2556 flags.go:64] FLAG: --v="2" Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003176 2556 flags.go:64] FLAG: --version="false" Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003181 2556 flags.go:64] FLAG: --vmodule="" Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003184 2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.003188 2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003324 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003328 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003331 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003334 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003337 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003340 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003342 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003345 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003348 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003351 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003353 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003356 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003358 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003361 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:56:11.011671 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003363 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003366 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003370 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003373 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003376 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003378 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003381 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003384 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003387 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003389 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003391 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003394 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003397 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003400 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003405 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003408 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003411 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003414 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003422 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:56:11.012198 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003425 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003428 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003431 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003433 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003436 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003438 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003441 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003444 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003447 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003449 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003452 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003454 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003457 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003460 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003462 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003466 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003468 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003471 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003474 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003476 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:56:11.012668 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003479 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003482 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003485 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003487 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003490 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003493 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003495 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003498 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003501 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003503 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003506 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003510 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003512 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003515 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003517 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003520 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003522 2556 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003525 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003528 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003532 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:56:11.013172 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003535 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003537 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003540 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003542 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003545 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003547 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003550 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003554 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003556 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003559 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003562 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003565 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003567 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003570 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003573 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003576 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003579 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003581 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003584 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003586 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:56:11.013659 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003589 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003591 2556 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003594 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003598 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003600 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.003603 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.004306 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.011543 2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.011558 2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.011581 2556 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.011590 2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011648 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011653 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011656 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011660 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:56:11.014186 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011663 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011665 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011668 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011672 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011675 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011678 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011681 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011683 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011687 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011691 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011695 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011699 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011701 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011704 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011707 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011710 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011713 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011715 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011718 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:56:11.014563 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011734 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011737 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011740 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011742 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011745 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011748 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011752 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011755 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011758 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011760 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011763 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011766 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011769 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011771 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011774 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011777 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011779 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011782 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011784 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:56:11.015091 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011787 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011789 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011792 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011794 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011797 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011800 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011802 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011804 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011807 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011809 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011812 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011814 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011817 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011819 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011822 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011824 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011827 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011830 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011832 2556 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011835 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:56:11.015571 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011838 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011841 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011844 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011846 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011849 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011852 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011854 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011857 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011859 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011862 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011864 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011868 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011872 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011874 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011877 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011879 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011882 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011885 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011887 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011889 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:56:11.016173 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011892 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011894 2556 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011897 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011900 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011902 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011905 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011907 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011910 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011912 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011915 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011917 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011920 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011923 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011926 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011928 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011931 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:56:11.016657 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.011935 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.011940 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012054 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012059 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012062 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012065 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012068 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012071 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012074 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012077 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012080 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012084 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012087 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012090 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012093 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:56:11.017065 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012096 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012099 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012102 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012104 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012107 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012109 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012111 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012114 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012117 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012119 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012123 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012126 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012129 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012133 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012135 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012139 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012141 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012144 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012146 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012148 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:56:11.017427 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012151 2556 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012154 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012156 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012158 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012161 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012163 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012166 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012169 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012171 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012174 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012176 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012179 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012181 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012184 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012186 2556 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012189 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012191 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012193 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012196 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012199 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:56:11.017946 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012201 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012204 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012207 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012209 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012212 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012215 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012218 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012220 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012224 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012226 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012229 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012231 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012234 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012237 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012239 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012242 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012244 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012247 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012250 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012252 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:56:11.018422 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012255 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012257 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012260 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012262 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012265 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012267 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012270 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012274 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012277 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012279 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012283 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012285 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012288 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012291 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012293 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012296 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012298 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012301 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012303 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012306 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:56:11.018918 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012308 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012311 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012314 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012316 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012319 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:11.012321 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.012326 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.012435 2556 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.014946 2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.016695 2556 server.go:1019] "Starting client certificate rotation" Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.016821 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 15:56:11.019404 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.016855 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 15:56:11.019864 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.019491 2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 15:56:11.025126 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.025107 2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 15:56:11.037055 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.037031 2556 log.go:25] "Validated CRI v1 runtime API" Aug 04 15:56:11.041031 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.041009 2556 log.go:25] "Validated CRI v1 image API" Aug 04 15:56:11.041925 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.041884 2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 15:56:11.044682 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.044654 2556 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8f263121-84ec-4abe-8215-15cea69cd6c8:/dev/nvme0n1p4 a181d3bd-d2f6-4eb9-90cb-83496d2f9c31:/dev/nvme0n1p3] Aug 04 15:56:11.044785 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.044681 2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 15:56:11.050667 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.050526 2556 manager.go:216] Machine: {Timestamp:2026-08-04 15:56:11.049765274 +0000 UTC m=+0.372188745 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099801 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24e63c97bf822be923230e37aad024 SystemUUID:ec24e63c-97bf-822b-e923-230e37aad024 BootID:7eaeb6ef-6758-43c9-9f9a-83c5b06c2678 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:86:9e:92:4c:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:86:9e:92:4c:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:4d:86:08:eb:8e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 15:56:11.050667 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.050666 2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 15:56:11.050818 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.050783 2556 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 15:56:11.054367 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.054341 2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 15:56:11.054510 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.054371 2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-227.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 15:56:11.055250 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.055239 2556 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 15:56:11.055290 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.055252 2556 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 15:56:11.055290 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.055260 2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 15:56:11.055290 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.055271 2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 15:56:11.056607 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.056594 2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 15:56:11.061945 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.061917 2556 state_mem.go:36] "Initialized new in-memory state store" Aug 04 15:56:11.062078 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.062068 2556 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 15:56:11.062969 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.062959 2556 kubelet.go:486] "Attempting to sync node with API server" Aug 04 15:56:11.063005 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.062974 2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 15:56:11.063005 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.062990 2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 15:56:11.063005 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.062999 2556 kubelet.go:398] "Adding apiserver pod source" Aug 04 15:56:11.063085 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.063008 2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 15:56:11.063412 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.063401 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 15:56:11.063456 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.063418 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 15:56:11.065814 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.065792 2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 15:56:11.068000 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.067983 2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 15:56:11.068094 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.068023 2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 15:56:11.069013 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.068997 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 15:56:11.075982 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.075954 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 15:56:11.076064 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.076040 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-227.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 15:56:11.170803 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170779 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 15:56:11.170803 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170803 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 15:56:11.170803 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170809 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 15:56:11.170803 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170815 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 15:56:11.170803 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170824 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 15:56:11.171059 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170830 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 15:56:11.171059 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170839 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 15:56:11.171059 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170844 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 15:56:11.171059 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170850 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 15:56:11.171059 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170856 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 15:56:11.171059 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170872 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 15:56:11.171059 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.170878 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 15:56:11.171861 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.171838 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 15:56:11.171861 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.171850 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 15:56:11.174361 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.174332 2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-227.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 15:56:11.175080 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.175064 2556 server.go:1268] "Started kubelet" Aug 04 15:56:11.175210 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.175156 2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 15:56:11.175335 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.175261 2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 15:56:11.175445 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.175352 2556 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 15:56:11.176031 ip-10-0-132-227 systemd[1]: Started Kubernetes Kubelet. Aug 04 15:56:11.180838 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.179449 2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 15:56:11.182251 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.182159 2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 15:56:11.182566 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.182549 2556 server.go:310] "Adding debug handlers to kubelet server" Aug 04 15:56:11.192831 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.192807 2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 15:56:11.192950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.192830 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 15:56:11.192950 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.192926 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:11.193672 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.193647 2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 15:56:11.193786 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.193669 2556 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 15:56:11.193786 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.193742 2556 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 15:56:11.193902 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.193836 2556 reconstruct.go:97] "Volume reconstruction finished" Aug 04 15:56:11.193902 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.193844 2556 reconciler.go:29] "Reconciler: start to sync state" Aug 04 15:56:11.194087 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.194073 2556 factory.go:55] Registering systemd factory Aug 04 15:56:11.194152 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.194092 2556 factory.go:223] Registration of the systemd container factory successfully Aug 04 15:56:11.194330 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.194317 2556 factory.go:153] Registering CRI-O factory Aug 04 15:56:11.194384 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.194334 2556 factory.go:223] Registration of the crio container factory successfully Aug 04 15:56:11.194437 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.194388 2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 15:56:11.194437 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.194420 2556 factory.go:103] Registering Raw factory Aug 04 15:56:11.194516 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.194440 2556 manager.go:1196] Started watching for new ooms in manager Aug 04 15:56:11.195054 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.195039 2556 manager.go:349] Starting recovery of all containers Aug 04 15:56:11.203379 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.203355 2556 manager.go:354] Recovery completed Aug 04 15:56:11.204430 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.204401 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 04 15:56:11.204539 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.204515 2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-227.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 04 15:56:11.206355 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.205187 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-227.ec2.internal.18c8a545bbd521c2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-227.ec2.internal,UID:ip-10-0-132-227.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-227.ec2.internal,},FirstTimestamp:2026-08-04 15:56:11.17503533 +0000 UTC m=+0.497458801,LastTimestamp:2026-08-04 15:56:11.17503533 +0000 UTC m=+0.497458801,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-227.ec2.internal,}" Aug 04 15:56:11.208123 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.208110 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:56:11.210101 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.210084 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:56:11.210154 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.210116 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:56:11.210154 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.210128 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:56:11.210664 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.210650 2556 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 15:56:11.210664 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.210656 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:56:11.210664 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.210664 2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 15:56:11.210802 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.210684 2556 state_mem.go:36] "Initialized new in-memory state store" Aug 04 15:56:11.211368 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.211354 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:56:11.211423 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.211379 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:56:11.211423 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.211391 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:56:11.212959 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.212946 2556 policy_none.go:49] "None policy: Start" Aug 04 15:56:11.213001 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.212965 2556 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 15:56:11.213001 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.212979 2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 15:56:11.215114 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.215102 2556 policy_none.go:47] "Start" Aug 04 15:56:11.215867 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.215772 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-227.ec2.internal.18c8a545bdec31a3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-227.ec2.internal,UID:ip-10-0-132-227.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-227.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-227.ec2.internal,},FirstTimestamp:2026-08-04 15:56:11.210101155 +0000 UTC m=+0.532524626,LastTimestamp:2026-08-04 15:56:11.210101155 +0000 UTC m=+0.532524626,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-227.ec2.internal,}" Aug 04 15:56:11.232484 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.232459 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jpxrt" Aug 04 15:56:11.236118 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.235851 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-227.ec2.internal.18c8a545bdec84d8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-227.ec2.internal,UID:ip-10-0-132-227.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-227.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-227.ec2.internal,},FirstTimestamp:2026-08-04 15:56:11.210122456 +0000 UTC m=+0.532545928,LastTimestamp:2026-08-04 15:56:11.210122456 +0000 UTC m=+0.532545928,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-227.ec2.internal,}" Aug 04 15:56:11.243917 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.243793 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jpxrt" Aug 04 15:56:11.249697 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.249628 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-227.ec2.internal.18c8a545bdecac67 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-227.ec2.internal,UID:ip-10-0-132-227.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-227.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-227.ec2.internal,},FirstTimestamp:2026-08-04 15:56:11.210132583 +0000 UTC m=+0.532556054,LastTimestamp:2026-08-04 15:56:11.210132583 +0000 UTC m=+0.532556054,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-227.ec2.internal,}" Aug 04 15:56:11.256375 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.256360 2556 manager.go:337] "Starting Device Plugin manager" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.256392 2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.256402 2556 server.go:87] "Starting device plugin registration server" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.256664 2556 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.256677 2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.256756 2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.256828 2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.256835 2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.257421 2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.257461 2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.265246 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.266431 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.266466 2556 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.266487 2556 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 15:56:11.269675 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.266523 2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 15:56:11.270139 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.269862 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 15:56:11.357316 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.357282 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:56:11.358313 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.358294 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:56:11.358417 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.358326 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:56:11.358417 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.358336 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:56:11.358417 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.358360 2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.367068 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.367047 2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal"] Aug 04 15:56:11.367119 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.367113 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:56:11.368694 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.368680 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:56:11.368762 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.368707 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:56:11.368762 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.368718 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:56:11.371111 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.371098 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:56:11.371212 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.371199 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.371255 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.371248 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:56:11.371855 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.371841 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:56:11.371928 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.371867 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:56:11.371928 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.371879 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:56:11.371928 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.371841 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:56:11.372037 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.371936 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:56:11.372037 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.371946 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:56:11.374175 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.374160 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.374240 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.374186 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:56:11.374240 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.374234 2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.374328 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.374252 2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-227.ec2.internal\": node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:11.374888 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.374874 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:56:11.374950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.374900 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:56:11.374950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.374911 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:56:11.394040 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.394014 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef5790e9892603695c64fc93d2f5fff7-config\") pod \"kube-apiserver-proxy-ip-10-0-132-227.ec2.internal\" (UID: \"ef5790e9892603695c64fc93d2f5fff7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.394195 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.394057 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e555b7abb158fbff1940921013c1e3b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal\" (UID: \"1e555b7abb158fbff1940921013c1e3b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.394195 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.394086 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e555b7abb158fbff1940921013c1e3b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal\" (UID: \"1e555b7abb158fbff1940921013c1e3b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.399348 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.399328 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:11.404520 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.404504 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-227.ec2.internal\" not found" node="ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.409035 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.409016 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-227.ec2.internal\" not found" node="ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.494562 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.494469 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e555b7abb158fbff1940921013c1e3b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal\" (UID: \"1e555b7abb158fbff1940921013c1e3b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.494562 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.494511 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e555b7abb158fbff1940921013c1e3b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal\" (UID: \"1e555b7abb158fbff1940921013c1e3b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.494562 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.494528 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef5790e9892603695c64fc93d2f5fff7-config\") pod \"kube-apiserver-proxy-ip-10-0-132-227.ec2.internal\" (UID: \"ef5790e9892603695c64fc93d2f5fff7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.494802 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.494570 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef5790e9892603695c64fc93d2f5fff7-config\") pod \"kube-apiserver-proxy-ip-10-0-132-227.ec2.internal\" (UID: \"ef5790e9892603695c64fc93d2f5fff7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.494802 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.494568 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e555b7abb158fbff1940921013c1e3b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal\" (UID: \"1e555b7abb158fbff1940921013c1e3b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.494802 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.494569 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e555b7abb158fbff1940921013c1e3b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal\" (UID: \"1e555b7abb158fbff1940921013c1e3b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.499590 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.499556 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:11.600346 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.600308 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:11.700635 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.700591 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:11.708325 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.708295 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.713024 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:11.712997 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal" Aug 04 15:56:11.801266 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.801170 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:11.901704 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:11.901667 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:12.002230 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:12.002192 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:12.015989 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.015961 2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 15:56:12.016148 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.016131 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 15:56:12.102854 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:12.102759 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:12.202958 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:12.202918 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:12.249073 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.249027 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 15:51:12 +0000 UTC" deadline="2028-05-05 10:15:21.14599486 +0000 UTC" Aug 04 15:56:12.249073 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.249064 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15354h19m8.896934787s" Aug 04 15:56:12.303336 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:12.303303 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:12.404324 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:12.404302 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-227.ec2.internal\" not found" Aug 04 15:56:12.413891 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.413867 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:56:12.415919 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.415896 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 15:56:12.416319 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.416297 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:56:12.416411 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.416336 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:56:12.416411 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.416351 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:56:12.417371 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:12.417321 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e555b7abb158fbff1940921013c1e3b.slice/crio-1f92d19e8bb3cfef412a05053e34402ede8d310d64175ac6e83fa0ecb45eace9 WatchSource:0}: Error finding container 1f92d19e8bb3cfef412a05053e34402ede8d310d64175ac6e83fa0ecb45eace9: Status 404 returned error can't find the container with id 1f92d19e8bb3cfef412a05053e34402ede8d310d64175ac6e83fa0ecb45eace9 Aug 04 15:56:12.422566 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.422546 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 15:56:12.423871 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:12.423851 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5790e9892603695c64fc93d2f5fff7.slice/crio-02df3fa206b01027d1b21b84e1c7f3ed1e08fd4419fc6f13080eb7b26bd41171 WatchSource:0}: Error finding container 02df3fa206b01027d1b21b84e1c7f3ed1e08fd4419fc6f13080eb7b26bd41171: Status 404 returned error can't find the container with id 02df3fa206b01027d1b21b84e1c7f3ed1e08fd4419fc6f13080eb7b26bd41171 Aug 04 15:56:12.493489 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.493445 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" Aug 04 15:56:12.508410 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.508384 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 15:56:12.510646 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.510628 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal" Aug 04 15:56:12.521054 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.521034 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 15:56:12.662851 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.662740 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 15:56:12.798520 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:12.798483 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 15:56:13.042939 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.042848 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 15:56:13.065854 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.065827 2556 apiserver.go:52] "Watching apiserver" Aug 04 15:56:13.076165 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.076135 2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 15:56:13.077616 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.077581 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-p6slt","openshift-network-diagnostics/network-check-target-94gw4","openshift-network-operator/iptables-alerter-sdx54","kube-system/global-pull-secret-syncer-986t7","kube-system/konnectivity-agent-nrkz8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq","openshift-cluster-node-tuning-operator/tuned-62lcl","openshift-dns/node-resolver-scrxz","openshift-ovn-kubernetes/ovnkube-node-wrjbh","kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal","openshift-image-registry/node-ca-4ppxb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal","openshift-multus/multus-additional-cni-plugins-s8xlk","openshift-multus/multus-vzlhw"] Aug 04 15:56:13.080241 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.080222 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.082699 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.082661 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.083518 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.083341 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 15:56:13.083518 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.083391 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s2tnd\"" Aug 04 15:56:13.083518 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.083409 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:56:13.085041 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.085019 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.085497 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.085478 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 15:56:13.085600 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.085478 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 15:56:13.086161 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.085831 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 15:56:13.086161 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.085920 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5gkdk\"" Aug 04 15:56:13.086161 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.085999 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 15:56:13.086647 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.086484 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 15:56:13.087480 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.087460 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.087565 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.087497 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 15:56:13.087565 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.087542 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:13.087887 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.087870 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 15:56:13.087887 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.087880 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pcxr4\"" Aug 04 15:56:13.088028 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.087973 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:56:13.089997 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.089979 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:13.092968 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.092420 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 15:56:13.092968 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.092469 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 15:56:13.092968 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.092502 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.092968 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.092665 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2lqcl\"" Aug 04 15:56:13.095000 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.094957 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m5r5t\"" Aug 04 15:56:13.095362 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.095225 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 15:56:13.095362 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.095275 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.095733 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.095573 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 15:56:13.096063 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.096031 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 15:56:13.098106 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.098088 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:13.098226 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.098156 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:13.098226 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.098174 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 15:56:13.098381 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.098229 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 15:56:13.099340 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.099286 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 15:56:13.099340 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.099329 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 15:56:13.099496 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.099313 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6hmmr\"" Aug 04 15:56:13.099496 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.099334 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 15:56:13.099631 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.099311 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 15:56:13.101281 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.101263 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:13.101399 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.101372 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:13.104536 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.104512 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108302 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-run-openvswitch\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108350 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f56a917-750c-4649-82c3-1a23788d486a-host-slash\") pod \"iptables-alerter-sdx54\" (UID: \"1f56a917-750c-4649-82c3-1a23788d486a\") " pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108386 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-run\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108420 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-device-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108492 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-systemd-units\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108519 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-run-ovn\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108550 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ad081f0-d1a9-4890-b1e9-4784a2f71d2a-agent-certs\") pod \"konnectivity-agent-nrkz8\" (UID: \"7ad081f0-d1a9-4890-b1e9-4784a2f71d2a\") " pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108570 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108581 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-sysctl-conf\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108615 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108646 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ad081f0-d1a9-4890-b1e9-4784a2f71d2a-konnectivity-ca\") pod \"konnectivity-agent-nrkz8\" (UID: \"7ad081f0-d1a9-4890-b1e9-4784a2f71d2a\") " pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108674 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvft5\" (UniqueName: \"kubernetes.io/projected/f175e9b5-253c-420c-bebc-ce9fc78e82dc-kube-api-access-cvft5\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108715 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d05e8f95-eabc-4eb2-8f47-116955bc09a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108748 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108780 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-kubernetes\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108835 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-registration-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.109373 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108864 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl2ml\" (UniqueName: \"kubernetes.io/projected/d05e8f95-eabc-4eb2-8f47-116955bc09a0-kube-api-access-dl2ml\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108875 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ftp6n\"" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108903 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-modprobe-d\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.108999 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-sys\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109037 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-etc-selinux\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109068 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-etc-openvswitch\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109158 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f175e9b5-253c-420c-bebc-ce9fc78e82dc-ovnkube-script-lib\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109211 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-sysconfig\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109304 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-lib-modules\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109343 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-host\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109457 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-kubelet\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109516 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-node-log\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109555 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-log-socket\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109631 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-cni-bin\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109695 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f175e9b5-253c-420c-bebc-ce9fc78e82dc-ovnkube-config\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109742 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f175e9b5-253c-420c-bebc-ce9fc78e82dc-env-overrides\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109792 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-system-cni-dir\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109856 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.110344 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109925 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b9f5fe66-9376-47a2-b714-de84ffb90ad8-dbus\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.109995 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110030 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-sys-fs\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110070 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7kwb\" (UniqueName: \"kubernetes.io/projected/3ad51ce2-face-404d-af19-4c0bfe22c59a-kube-api-access-c7kwb\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110127 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-cnibin\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110160 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef8a69b0-b53d-4781-9265-5792c4aae2f0-tmp\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110198 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-run-systemd\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110232 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110277 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-sysctl-d\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110361 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-run-netns\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110560 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-cni-netd\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110605 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110659 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-tuned\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110812 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-socket-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110861 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f175e9b5-253c-420c-bebc-ce9fc78e82dc-ovn-node-metrics-cert\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110897 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-os-release\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.111254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.110936 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f56a917-750c-4649-82c3-1a23788d486a-iptables-alerter-script\") pod \"iptables-alerter-sdx54\" (UID: \"1f56a917-750c-4649-82c3-1a23788d486a\") " pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.112019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.111043 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d05e8f95-eabc-4eb2-8f47-116955bc09a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.112019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.111078 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d05e8f95-eabc-4eb2-8f47-116955bc09a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.112019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.111109 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-systemd\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.112019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.111142 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-var-lib-openvswitch\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.112019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.111180 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b9f5fe66-9376-47a2-b714-de84ffb90ad8-kubelet-config\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.112019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.111211 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4q7r\" (UniqueName: \"kubernetes.io/projected/1f56a917-750c-4649-82c3-1a23788d486a-kube-api-access-x4q7r\") pod \"iptables-alerter-sdx54\" (UID: \"1f56a917-750c-4649-82c3-1a23788d486a\") " pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.112019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.111441 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-var-lib-kubelet\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.112019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.111489 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5fqv\" (UniqueName: \"kubernetes.io/projected/ef8a69b0-b53d-4781-9265-5792c4aae2f0-kube-api-access-k5fqv\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.112019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.111527 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-slash\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.112516 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.112469 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 15:56:13.112641 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.112605 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.113120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.113102 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 15:56:13.113455 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.113435 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rxqrm\"" Aug 04 15:56:13.113605 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.113494 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 15:56:13.115179 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.115160 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zvgfv\"" Aug 04 15:56:13.115281 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.115225 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 15:56:13.115634 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.115612 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 15:56:13.194072 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.194045 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 15:56:13.194915 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.194893 2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 15:56:13.209171 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.209144 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 15:56:13.212576 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212553 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-host\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.212719 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212585 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-kubelet\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212719 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212612 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-node-log\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212719 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212635 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-log-socket\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212719 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212679 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-cni-bin\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212712 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-kubelet\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212714 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-host\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212717 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrdx7\" (UniqueName: \"kubernetes.io/projected/4e47afd1-3679-47c0-bbdb-66ef0145e925-kube-api-access-xrdx7\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212769 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-log-socket\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212795 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-node-log\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212844 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-cni-bin\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212855 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f175e9b5-253c-420c-bebc-ce9fc78e82dc-ovnkube-config\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212891 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f175e9b5-253c-420c-bebc-ce9fc78e82dc-env-overrides\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212919 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-system-cni-dir\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.212950 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212946 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b9f5fe66-9376-47a2-b714-de84ffb90ad8-dbus\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212971 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212984 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-system-cni-dir\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.212996 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-sys-fs\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213047 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-sys-fs\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213060 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7kwb\" (UniqueName: \"kubernetes.io/projected/3ad51ce2-face-404d-af19-4c0bfe22c59a-kube-api-access-c7kwb\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213102 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-cnibin\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213128 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef8a69b0-b53d-4781-9265-5792c4aae2f0-tmp\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.213129 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213156 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-run-systemd\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213223 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b9f5fe66-9376-47a2-b714-de84ffb90ad8-dbus\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213237 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-cnibin\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213242 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-run-systemd\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.213287 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret podName:b9f5fe66-9376-47a2-b714-de84ffb90ad8 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:13.713198859 +0000 UTC m=+3.035622335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret") pod "global-pull-secret-syncer-986t7" (UID: "b9f5fe66-9376-47a2-b714-de84ffb90ad8") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213343 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213374 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-os-release\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213402 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-var-lib-kubelet\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.213439 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213428 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-etc-kubernetes\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213457 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-sysctl-d\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213455 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213484 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77cfdd06-8d08-4813-848f-43139a3c05b7-cni-binary-copy\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213523 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-run-netns\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213548 2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213564 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-var-lib-cni-bin\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213562 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f175e9b5-253c-420c-bebc-ce9fc78e82dc-ovnkube-config\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213600 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-var-lib-cni-multus\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213630 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-hostroot\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213633 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-sysctl-d\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213657 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/de8b7d3f-9598-47f7-a387-8117c80acc3c-hosts-file\") pod \"node-resolver-scrxz\" (UID: \"de8b7d3f-9598-47f7-a387-8117c80acc3c\") " pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213683 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8b7d3f-9598-47f7-a387-8117c80acc3c-tmp-dir\") pod \"node-resolver-scrxz\" (UID: \"de8b7d3f-9598-47f7-a387-8117c80acc3c\") " pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213712 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-run-netns\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213741 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-cni-netd\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213788 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213816 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-tuned\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213811 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-run-netns\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.214350 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213841 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-socket-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213868 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f175e9b5-253c-420c-bebc-ce9fc78e82dc-ovn-node-metrics-cert\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213874 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-cni-netd\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213893 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-os-release\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213940 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f56a917-750c-4649-82c3-1a23788d486a-iptables-alerter-script\") pod \"iptables-alerter-sdx54\" (UID: \"1f56a917-750c-4649-82c3-1a23788d486a\") " pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213967 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blbqq\" (UniqueName: \"kubernetes.io/projected/77cfdd06-8d08-4813-848f-43139a3c05b7-kube-api-access-blbqq\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213970 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f175e9b5-253c-420c-bebc-ce9fc78e82dc-env-overrides\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213992 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6484c73f-9461-43ec-bf2a-924665d325a6-serviceca\") pod \"node-ca-4ppxb\" (UID: \"6484c73f-9461-43ec-bf2a-924665d325a6\") " pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214071 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-socket-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214078 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-os-release\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.213874 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214124 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzwqr\" (UniqueName: \"kubernetes.io/projected/6484c73f-9461-43ec-bf2a-924665d325a6-kube-api-access-vzwqr\") pod \"node-ca-4ppxb\" (UID: \"6484c73f-9461-43ec-bf2a-924665d325a6\") " pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214200 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-run-multus-certs\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214228 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d05e8f95-eabc-4eb2-8f47-116955bc09a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214281 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d05e8f95-eabc-4eb2-8f47-116955bc09a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214323 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-systemd\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214349 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-var-lib-openvswitch\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.215259 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214375 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b9f5fe66-9376-47a2-b714-de84ffb90ad8-kubelet-config\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214422 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4q7r\" (UniqueName: \"kubernetes.io/projected/1f56a917-750c-4649-82c3-1a23788d486a-kube-api-access-x4q7r\") pod \"iptables-alerter-sdx54\" (UID: \"1f56a917-750c-4649-82c3-1a23788d486a\") " pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214427 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-systemd\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214465 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-cni-dir\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214494 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b9f5fe66-9376-47a2-b714-de84ffb90ad8-kubelet-config\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214493 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-var-lib-kubelet\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214538 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-var-lib-openvswitch\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214541 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5fqv\" (UniqueName: \"kubernetes.io/projected/ef8a69b0-b53d-4781-9265-5792c4aae2f0-kube-api-access-k5fqv\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214578 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-slash\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214601 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-run-openvswitch\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214625 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f56a917-750c-4649-82c3-1a23788d486a-host-slash\") pod \"iptables-alerter-sdx54\" (UID: \"1f56a917-750c-4649-82c3-1a23788d486a\") " pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214656 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214682 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-run-k8s-cni-cncf-io\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214737 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-run\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214780 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-device-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214798 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f56a917-750c-4649-82c3-1a23788d486a-host-slash\") pod \"iptables-alerter-sdx54\" (UID: \"1f56a917-750c-4649-82c3-1a23788d486a\") " pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214803 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-systemd-units\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.216120 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214858 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-run-ovn\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214894 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ad081f0-d1a9-4890-b1e9-4784a2f71d2a-agent-certs\") pod \"konnectivity-agent-nrkz8\" (UID: \"7ad081f0-d1a9-4890-b1e9-4784a2f71d2a\") " pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214912 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-systemd-units\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214720 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f56a917-750c-4649-82c3-1a23788d486a-iptables-alerter-script\") pod \"iptables-alerter-sdx54\" (UID: \"1f56a917-750c-4649-82c3-1a23788d486a\") " pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214932 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6484c73f-9461-43ec-bf2a-924665d325a6-host\") pod \"node-ca-4ppxb\" (UID: \"6484c73f-9461-43ec-bf2a-924665d325a6\") " pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214968 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-var-lib-kubelet\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214981 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d05e8f95-eabc-4eb2-8f47-116955bc09a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215037 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-run-openvswitch\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.214982 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d05e8f95-eabc-4eb2-8f47-116955bc09a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215051 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-run\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215075 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-run-ovn\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215120 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-host-slash\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215163 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-sysctl-conf\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215166 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-device-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215211 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215249 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-conf-dir\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215276 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-daemon-config\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.217035 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215292 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-sysctl-conf\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215301 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ad081f0-d1a9-4890-b1e9-4784a2f71d2a-konnectivity-ca\") pod \"konnectivity-agent-nrkz8\" (UID: \"7ad081f0-d1a9-4890-b1e9-4784a2f71d2a\") " pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215347 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215359 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215424 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvft5\" (UniqueName: \"kubernetes.io/projected/f175e9b5-253c-420c-bebc-ce9fc78e82dc-kube-api-access-cvft5\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215491 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d05e8f95-eabc-4eb2-8f47-116955bc09a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215762 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215799 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c92jw\" (UniqueName: \"kubernetes.io/projected/de8b7d3f-9598-47f7-a387-8117c80acc3c-kube-api-access-c92jw\") pod \"node-resolver-scrxz\" (UID: \"de8b7d3f-9598-47f7-a387-8117c80acc3c\") " pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215839 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-kubernetes\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215878 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-registration-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215909 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl2ml\" (UniqueName: \"kubernetes.io/projected/d05e8f95-eabc-4eb2-8f47-116955bc09a0-kube-api-access-dl2ml\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215937 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-system-cni-dir\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.215989 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-modprobe-d\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216014 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-sys\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216037 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-etc-selinux\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216061 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-etc-openvswitch\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216086 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f175e9b5-253c-420c-bebc-ce9fc78e82dc-ovnkube-script-lib\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.217885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216110 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-cnibin\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216138 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-socket-dir-parent\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216209 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ad081f0-d1a9-4890-b1e9-4784a2f71d2a-konnectivity-ca\") pod \"konnectivity-agent-nrkz8\" (UID: \"7ad081f0-d1a9-4890-b1e9-4784a2f71d2a\") " pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216212 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-sysconfig\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216260 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-sysconfig\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216289 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-lib-modules\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216484 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-lib-modules\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216535 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-kubernetes\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216573 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-modprobe-d\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216628 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-registration-dir\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216683 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d05e8f95-eabc-4eb2-8f47-116955bc09a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216755 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d05e8f95-eabc-4eb2-8f47-116955bc09a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216956 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f175e9b5-253c-420c-bebc-ce9fc78e82dc-etc-openvswitch\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.216959 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3ad51ce2-face-404d-af19-4c0bfe22c59a-etc-selinux\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.217572 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ef8a69b0-b53d-4781-9265-5792c4aae2f0-sys\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.217581 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f175e9b5-253c-420c-bebc-ce9fc78e82dc-ovnkube-script-lib\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.218106 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef8a69b0-b53d-4781-9265-5792c4aae2f0-tmp\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.218654 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.218134 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ef8a69b0-b53d-4781-9265-5792c4aae2f0-etc-tuned\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.219256 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.218687 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ad081f0-d1a9-4890-b1e9-4784a2f71d2a-agent-certs\") pod \"konnectivity-agent-nrkz8\" (UID: \"7ad081f0-d1a9-4890-b1e9-4784a2f71d2a\") " pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:13.219687 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.219663 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f175e9b5-253c-420c-bebc-ce9fc78e82dc-ovn-node-metrics-cert\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.222931 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.222905 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7kwb\" (UniqueName: \"kubernetes.io/projected/3ad51ce2-face-404d-af19-4c0bfe22c59a-kube-api-access-c7kwb\") pod \"aws-ebs-csi-driver-node-q4gpq\" (UID: \"3ad51ce2-face-404d-af19-4c0bfe22c59a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.225789 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.225739 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4q7r\" (UniqueName: \"kubernetes.io/projected/1f56a917-750c-4649-82c3-1a23788d486a-kube-api-access-x4q7r\") pod \"iptables-alerter-sdx54\" (UID: \"1f56a917-750c-4649-82c3-1a23788d486a\") " pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.226265 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.226217 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5fqv\" (UniqueName: \"kubernetes.io/projected/ef8a69b0-b53d-4781-9265-5792c4aae2f0-kube-api-access-k5fqv\") pod \"tuned-62lcl\" (UID: \"ef8a69b0-b53d-4781-9265-5792c4aae2f0\") " pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.226459 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.226438 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvft5\" (UniqueName: \"kubernetes.io/projected/f175e9b5-253c-420c-bebc-ce9fc78e82dc-kube-api-access-cvft5\") pod \"ovnkube-node-wrjbh\" (UID: \"f175e9b5-253c-420c-bebc-ce9fc78e82dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.230369 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.230348 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl2ml\" (UniqueName: \"kubernetes.io/projected/d05e8f95-eabc-4eb2-8f47-116955bc09a0-kube-api-access-dl2ml\") pod \"multus-additional-cni-plugins-s8xlk\" (UID: \"d05e8f95-eabc-4eb2-8f47-116955bc09a0\") " pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.238249 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.238220 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-92q8t" Aug 04 15:56:13.249763 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.249608 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-92q8t" Aug 04 15:56:13.275354 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.275288 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal" event={"ID":"ef5790e9892603695c64fc93d2f5fff7","Type":"ContainerStarted","Data":"02df3fa206b01027d1b21b84e1c7f3ed1e08fd4419fc6f13080eb7b26bd41171"} Aug 04 15:56:13.277508 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.277475 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" event={"ID":"1e555b7abb158fbff1940921013c1e3b","Type":"ContainerStarted","Data":"1f92d19e8bb3cfef412a05053e34402ede8d310d64175ac6e83fa0ecb45eace9"} Aug 04 15:56:13.317761 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317659 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-cnibin\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.317761 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317702 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-socket-dir-parent\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.317761 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317736 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdx7\" (UniqueName: \"kubernetes.io/projected/4e47afd1-3679-47c0-bbdb-66ef0145e925-kube-api-access-xrdx7\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:13.318007 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317797 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-os-release\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318007 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317824 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-socket-dir-parent\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318007 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317820 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-cnibin\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318007 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317824 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-var-lib-kubelet\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318007 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317869 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-var-lib-kubelet\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318007 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317893 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-etc-kubernetes\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318007 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317919 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77cfdd06-8d08-4813-848f-43139a3c05b7-cni-binary-copy\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318007 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317943 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-run-netns\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318007 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.317966 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-var-lib-cni-bin\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318021 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-var-lib-cni-multus\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318046 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-hostroot\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318070 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/de8b7d3f-9598-47f7-a387-8117c80acc3c-hosts-file\") pod \"node-resolver-scrxz\" (UID: \"de8b7d3f-9598-47f7-a387-8117c80acc3c\") " pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318093 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8b7d3f-9598-47f7-a387-8117c80acc3c-tmp-dir\") pod \"node-resolver-scrxz\" (UID: \"de8b7d3f-9598-47f7-a387-8117c80acc3c\") " pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318126 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blbqq\" (UniqueName: \"kubernetes.io/projected/77cfdd06-8d08-4813-848f-43139a3c05b7-kube-api-access-blbqq\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318163 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-var-lib-cni-bin\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318171 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-os-release\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318177 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6484c73f-9461-43ec-bf2a-924665d325a6-serviceca\") pod \"node-ca-4ppxb\" (UID: \"6484c73f-9461-43ec-bf2a-924665d325a6\") " pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318202 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzwqr\" (UniqueName: \"kubernetes.io/projected/6484c73f-9461-43ec-bf2a-924665d325a6-kube-api-access-vzwqr\") pod \"node-ca-4ppxb\" (UID: \"6484c73f-9461-43ec-bf2a-924665d325a6\") " pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318219 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-etc-kubernetes\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318231 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-run-multus-certs\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318261 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-cni-dir\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318289 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318312 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-run-k8s-cni-cncf-io\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318337 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6484c73f-9461-43ec-bf2a-924665d325a6-host\") pod \"node-ca-4ppxb\" (UID: \"6484c73f-9461-43ec-bf2a-924665d325a6\") " pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318362 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-conf-dir\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318385 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-daemon-config\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.318429 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318413 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318440 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c92jw\" (UniqueName: \"kubernetes.io/projected/de8b7d3f-9598-47f7-a387-8117c80acc3c-kube-api-access-c92jw\") pod \"node-resolver-scrxz\" (UID: \"de8b7d3f-9598-47f7-a387-8117c80acc3c\") " pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318468 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-system-cni-dir\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318554 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-system-cni-dir\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318559 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8b7d3f-9598-47f7-a387-8117c80acc3c-tmp-dir\") pod \"node-resolver-scrxz\" (UID: \"de8b7d3f-9598-47f7-a387-8117c80acc3c\") " pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318617 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-var-lib-cni-multus\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318662 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-hostroot\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318713 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/de8b7d3f-9598-47f7-a387-8117c80acc3c-hosts-file\") pod \"node-resolver-scrxz\" (UID: \"de8b7d3f-9598-47f7-a387-8117c80acc3c\") " pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318818 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-run-netns\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318897 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-run-k8s-cni-cncf-io\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.318977 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6484c73f-9461-43ec-bf2a-924665d325a6-serviceca\") pod \"node-ca-4ppxb\" (UID: \"6484c73f-9461-43ec-bf2a-924665d325a6\") " pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.319032 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6484c73f-9461-43ec-bf2a-924665d325a6-host\") pod \"node-ca-4ppxb\" (UID: \"6484c73f-9461-43ec-bf2a-924665d325a6\") " pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.319097 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.319139 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-host-run-multus-certs\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.319169 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs podName:4e47afd1-3679-47c0-bbdb-66ef0145e925 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:13.819150207 +0000 UTC m=+3.141573670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs") pod "network-metrics-daemon-p6slt" (UID: "4e47afd1-3679-47c0-bbdb-66ef0145e925") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:13.319202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.319206 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-cni-dir\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319681 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.319285 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-conf-dir\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319681 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.319394 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77cfdd06-8d08-4813-848f-43139a3c05b7-multus-daemon-config\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.319681 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.319588 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77cfdd06-8d08-4813-848f-43139a3c05b7-cni-binary-copy\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.327485 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.327459 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:56:13.327485 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.327489 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:56:13.327685 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.327501 2556 projected.go:196] Error preparing data for projected volume kube-api-access-4dkcn for pod openshift-network-diagnostics/network-check-target-94gw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:13.327685 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.327586 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn podName:deaef137-9bcb-4e3b-b07e-4bd32c57ba2e nodeName:}" failed. No retries permitted until 2026-08-04 15:56:13.827564409 +0000 UTC m=+3.149987891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4dkcn" (UniqueName: "kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn") pod "network-check-target-94gw4" (UID: "deaef137-9bcb-4e3b-b07e-4bd32c57ba2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:13.329076 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.329043 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrdx7\" (UniqueName: \"kubernetes.io/projected/4e47afd1-3679-47c0-bbdb-66ef0145e925-kube-api-access-xrdx7\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:13.330363 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.330339 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzwqr\" (UniqueName: \"kubernetes.io/projected/6484c73f-9461-43ec-bf2a-924665d325a6-kube-api-access-vzwqr\") pod \"node-ca-4ppxb\" (UID: \"6484c73f-9461-43ec-bf2a-924665d325a6\") " pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.331166 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.331129 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blbqq\" (UniqueName: \"kubernetes.io/projected/77cfdd06-8d08-4813-848f-43139a3c05b7-kube-api-access-blbqq\") pod \"multus-vzlhw\" (UID: \"77cfdd06-8d08-4813-848f-43139a3c05b7\") " pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.331349 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.331310 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c92jw\" (UniqueName: \"kubernetes.io/projected/de8b7d3f-9598-47f7-a387-8117c80acc3c-kube-api-access-c92jw\") pod \"node-resolver-scrxz\" (UID: \"de8b7d3f-9598-47f7-a387-8117c80acc3c\") " pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.393078 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.393039 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62lcl" Aug 04 15:56:13.400923 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.400899 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" Aug 04 15:56:13.404364 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:13.404131 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8a69b0_b53d_4781_9265_5792c4aae2f0.slice/crio-6a805d63cc52b2f97e001d8621e30efdf74f24cfe4570dbcaf4f91be9e0bb32a WatchSource:0}: Error finding container 6a805d63cc52b2f97e001d8621e30efdf74f24cfe4570dbcaf4f91be9e0bb32a: Status 404 returned error can't find the container with id 6a805d63cc52b2f97e001d8621e30efdf74f24cfe4570dbcaf4f91be9e0bb32a Aug 04 15:56:13.412035 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:13.411994 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05e8f95_eabc_4eb2_8f47_116955bc09a0.slice/crio-883297eb4b4412d26eaa6aedb6f5202ec135998541d03a39b0a011ac600036d8 WatchSource:0}: Error finding container 883297eb4b4412d26eaa6aedb6f5202ec135998541d03a39b0a011ac600036d8: Status 404 returned error can't find the container with id 883297eb4b4412d26eaa6aedb6f5202ec135998541d03a39b0a011ac600036d8 Aug 04 15:56:13.416831 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.416803 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdx54" Aug 04 15:56:13.419697 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.419673 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:13.426537 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.426511 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" Aug 04 15:56:13.430780 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:13.430749 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f56a917_750c_4649_82c3_1a23788d486a.slice/crio-76635af0f14884e95b5466b17016e2790e486894f11b84c9f8ea0b7c02e0758d WatchSource:0}: Error finding container 76635af0f14884e95b5466b17016e2790e486894f11b84c9f8ea0b7c02e0758d: Status 404 returned error can't find the container with id 76635af0f14884e95b5466b17016e2790e486894f11b84c9f8ea0b7c02e0758d Aug 04 15:56:13.432269 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:13.432228 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad081f0_d1a9_4890_b1e9_4784a2f71d2a.slice/crio-7bb5528022fd5b4f1c972df596179ac6ee5308589a3b5e396ec8e636b7047015 WatchSource:0}: Error finding container 7bb5528022fd5b4f1c972df596179ac6ee5308589a3b5e396ec8e636b7047015: Status 404 returned error can't find the container with id 7bb5528022fd5b4f1c972df596179ac6ee5308589a3b5e396ec8e636b7047015 Aug 04 15:56:13.433026 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.433001 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:13.438779 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:13.438757 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad51ce2_face_404d_af19_4c0bfe22c59a.slice/crio-afedbf4176f038dee58349d2607f0c6e8d38ae592b2ed5d3408348a17ff120f4 WatchSource:0}: Error finding container afedbf4176f038dee58349d2607f0c6e8d38ae592b2ed5d3408348a17ff120f4: Status 404 returned error can't find the container with id afedbf4176f038dee58349d2607f0c6e8d38ae592b2ed5d3408348a17ff120f4 Aug 04 15:56:13.441491 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.441468 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vzlhw" Aug 04 15:56:13.444196 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:13.444171 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf175e9b5_253c_420c_bebc_ce9fc78e82dc.slice/crio-6e79512343fc2217dbe34130745b5fb5d9409baf3b1dc1c708e393be3d45a3f3 WatchSource:0}: Error finding container 6e79512343fc2217dbe34130745b5fb5d9409baf3b1dc1c708e393be3d45a3f3: Status 404 returned error can't find the container with id 6e79512343fc2217dbe34130745b5fb5d9409baf3b1dc1c708e393be3d45a3f3 Aug 04 15:56:13.449989 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.449969 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ppxb" Aug 04 15:56:13.451599 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:13.451573 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77cfdd06_8d08_4813_848f_43139a3c05b7.slice/crio-88937baf78e8196e3f5650b4c9f125bc87de20e082184c77917d7d04cb7bef70 WatchSource:0}: Error finding container 88937baf78e8196e3f5650b4c9f125bc87de20e082184c77917d7d04cb7bef70: Status 404 returned error can't find the container with id 88937baf78e8196e3f5650b4c9f125bc87de20e082184c77917d7d04cb7bef70 Aug 04 15:56:13.457034 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.457014 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-scrxz" Aug 04 15:56:13.459673 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:13.459652 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6484c73f_9461_43ec_bf2a_924665d325a6.slice/crio-466c26e8ec414f944c8b92d8b76555be3a408545735e306159f29d5f9d6be6b5 WatchSource:0}: Error finding container 466c26e8ec414f944c8b92d8b76555be3a408545735e306159f29d5f9d6be6b5: Status 404 returned error can't find the container with id 466c26e8ec414f944c8b92d8b76555be3a408545735e306159f29d5f9d6be6b5 Aug 04 15:56:13.467485 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:13.467450 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde8b7d3f_9598_47f7_a387_8117c80acc3c.slice/crio-e9b0a76760484668adc1f288c8d50b5b463034a47efea5bd1c15acff79b900b9 WatchSource:0}: Error finding container e9b0a76760484668adc1f288c8d50b5b463034a47efea5bd1c15acff79b900b9: Status 404 returned error can't find the container with id e9b0a76760484668adc1f288c8d50b5b463034a47efea5bd1c15acff79b900b9 Aug 04 15:56:13.721605 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.721516 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:13.721777 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.721695 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:13.721777 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.721776 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret podName:b9f5fe66-9376-47a2-b714-de84ffb90ad8 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:14.721759469 +0000 UTC m=+4.044182930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret") pod "global-pull-secret-syncer-986t7" (UID: "b9f5fe66-9376-47a2-b714-de84ffb90ad8") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:13.822405 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.822363 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:13.822601 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.822509 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:13.822601 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.822595 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs podName:4e47afd1-3679-47c0-bbdb-66ef0145e925 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:14.822572608 +0000 UTC m=+4.144996086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs") pod "network-metrics-daemon-p6slt" (UID: "4e47afd1-3679-47c0-bbdb-66ef0145e925") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:13.922840 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:13.922814 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:13.922992 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.922974 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:56:13.923062 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.922996 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:56:13.923062 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.923006 2556 projected.go:196] Error preparing data for projected volume kube-api-access-4dkcn for pod openshift-network-diagnostics/network-check-target-94gw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:13.923062 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:13.923056 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn podName:deaef137-9bcb-4e3b-b07e-4bd32c57ba2e nodeName:}" failed. No retries permitted until 2026-08-04 15:56:14.923042126 +0000 UTC m=+4.245465585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dkcn" (UniqueName: "kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn") pod "network-check-target-94gw4" (UID: "deaef137-9bcb-4e3b-b07e-4bd32c57ba2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:14.255029 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.254880 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 15:51:14 +0000 UTC" deadline="2028-04-20 17:37:28.324879782 +0000 UTC" Aug 04 15:56:14.255029 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.254918 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15001h41m14.069966262s" Aug 04 15:56:14.288981 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.288941 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-scrxz" event={"ID":"de8b7d3f-9598-47f7-a387-8117c80acc3c","Type":"ContainerStarted","Data":"e9b0a76760484668adc1f288c8d50b5b463034a47efea5bd1c15acff79b900b9"} Aug 04 15:56:14.294907 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.294866 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ppxb" event={"ID":"6484c73f-9461-43ec-bf2a-924665d325a6","Type":"ContainerStarted","Data":"466c26e8ec414f944c8b92d8b76555be3a408545735e306159f29d5f9d6be6b5"} Aug 04 15:56:14.299237 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.299200 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vzlhw" event={"ID":"77cfdd06-8d08-4813-848f-43139a3c05b7","Type":"ContainerStarted","Data":"88937baf78e8196e3f5650b4c9f125bc87de20e082184c77917d7d04cb7bef70"} Aug 04 15:56:14.303321 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.303192 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" event={"ID":"f175e9b5-253c-420c-bebc-ce9fc78e82dc","Type":"ContainerStarted","Data":"6e79512343fc2217dbe34130745b5fb5d9409baf3b1dc1c708e393be3d45a3f3"} Aug 04 15:56:14.305359 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.305244 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdx54" event={"ID":"1f56a917-750c-4649-82c3-1a23788d486a","Type":"ContainerStarted","Data":"76635af0f14884e95b5466b17016e2790e486894f11b84c9f8ea0b7c02e0758d"} Aug 04 15:56:14.314848 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.314766 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" event={"ID":"d05e8f95-eabc-4eb2-8f47-116955bc09a0","Type":"ContainerStarted","Data":"883297eb4b4412d26eaa6aedb6f5202ec135998541d03a39b0a011ac600036d8"} Aug 04 15:56:14.326493 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.326405 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62lcl" event={"ID":"ef8a69b0-b53d-4781-9265-5792c4aae2f0","Type":"ContainerStarted","Data":"6a805d63cc52b2f97e001d8621e30efdf74f24cfe4570dbcaf4f91be9e0bb32a"} Aug 04 15:56:14.328878 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.328770 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" event={"ID":"3ad51ce2-face-404d-af19-4c0bfe22c59a","Type":"ContainerStarted","Data":"afedbf4176f038dee58349d2607f0c6e8d38ae592b2ed5d3408348a17ff120f4"} Aug 04 15:56:14.333746 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.333656 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nrkz8" event={"ID":"7ad081f0-d1a9-4890-b1e9-4784a2f71d2a","Type":"ContainerStarted","Data":"7bb5528022fd5b4f1c972df596179ac6ee5308589a3b5e396ec8e636b7047015"} Aug 04 15:56:14.731184 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.730517 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:14.731184 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:14.730689 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:14.731184 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:14.730756 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret podName:b9f5fe66-9376-47a2-b714-de84ffb90ad8 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:16.730737201 +0000 UTC m=+6.053160667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret") pod "global-pull-secret-syncer-986t7" (UID: "b9f5fe66-9376-47a2-b714-de84ffb90ad8") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:14.831313 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.831030 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:14.831313 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:14.831309 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:14.831552 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:14.831393 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs podName:4e47afd1-3679-47c0-bbdb-66ef0145e925 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:16.831371201 +0000 UTC m=+6.153794675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs") pod "network-metrics-daemon-p6slt" (UID: "4e47afd1-3679-47c0-bbdb-66ef0145e925") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:14.931902 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:14.931795 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:14.932078 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:14.932041 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:56:14.932078 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:14.932071 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:56:14.932268 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:14.932086 2556 projected.go:196] Error preparing data for projected volume kube-api-access-4dkcn for pod openshift-network-diagnostics/network-check-target-94gw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:14.932268 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:14.932152 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn podName:deaef137-9bcb-4e3b-b07e-4bd32c57ba2e nodeName:}" failed. No retries permitted until 2026-08-04 15:56:16.93213144 +0000 UTC m=+6.254554903 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dkcn" (UniqueName: "kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn") pod "network-check-target-94gw4" (UID: "deaef137-9bcb-4e3b-b07e-4bd32c57ba2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:15.256090 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:15.255945 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 15:51:14 +0000 UTC" deadline="2028-04-17 11:49:26.293599502 +0000 UTC" Aug 04 15:56:15.256090 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:15.256016 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14923h53m11.037588285s" Aug 04 15:56:15.269529 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:15.269497 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:15.269714 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:15.269637 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:15.270962 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:15.270095 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:15.270962 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:15.270207 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:15.270962 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:15.270271 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:15.270962 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:15.270339 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:15.354885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:15.354715 2556 generic.go:356] "Generic (PLEG): container finished" podID="1e555b7abb158fbff1940921013c1e3b" containerID="95969756f617b9d5825d4a86f794a4cdefb67e0439779728633affed04cb6b7e" exitCode=0 Aug 04 15:56:15.354885 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:15.354802 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" event={"ID":"1e555b7abb158fbff1940921013c1e3b","Type":"ContainerDied","Data":"95969756f617b9d5825d4a86f794a4cdefb67e0439779728633affed04cb6b7e"} Aug 04 15:56:16.381354 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:16.381245 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" event={"ID":"1e555b7abb158fbff1940921013c1e3b","Type":"ContainerStarted","Data":"1883c50944627fa84c0f05b254c9635d6f5ac670151300f7639aba17b14179e0"} Aug 04 15:56:16.747304 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:16.747218 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:16.747520 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:16.747425 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:16.747520 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:16.747491 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret podName:b9f5fe66-9376-47a2-b714-de84ffb90ad8 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:20.747471883 +0000 UTC m=+10.069895348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret") pod "global-pull-secret-syncer-986t7" (UID: "b9f5fe66-9376-47a2-b714-de84ffb90ad8") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:16.847786 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:16.847733 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:16.847975 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:16.847910 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:16.847975 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:16.847962 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs podName:4e47afd1-3679-47c0-bbdb-66ef0145e925 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:20.847949475 +0000 UTC m=+10.170372939 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs") pod "network-metrics-daemon-p6slt" (UID: "4e47afd1-3679-47c0-bbdb-66ef0145e925") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:16.949177 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:16.949140 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:16.949390 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:16.949331 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:56:16.949390 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:16.949358 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:56:16.949390 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:16.949372 2556 projected.go:196] Error preparing data for projected volume kube-api-access-4dkcn for pod openshift-network-diagnostics/network-check-target-94gw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:16.949562 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:16.949438 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn podName:deaef137-9bcb-4e3b-b07e-4bd32c57ba2e nodeName:}" failed. No retries permitted until 2026-08-04 15:56:20.949417963 +0000 UTC m=+10.271841426 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dkcn" (UniqueName: "kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn") pod "network-check-target-94gw4" (UID: "deaef137-9bcb-4e3b-b07e-4bd32c57ba2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:17.267655 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:17.267619 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:17.267655 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:17.267641 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:17.268017 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:17.267780 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:17.268017 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:17.267868 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:17.268017 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:17.267925 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:17.268017 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:17.268003 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:19.267889 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:19.267760 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:19.267889 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:19.267794 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:19.269345 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:19.268854 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:19.269345 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:19.268986 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:19.269345 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:19.268997 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:19.269345 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:19.269311 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:20.783284 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:20.783242 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:20.783814 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:20.783406 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:20.783814 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:20.783473 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret podName:b9f5fe66-9376-47a2-b714-de84ffb90ad8 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:28.783452037 +0000 UTC m=+18.105875524 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret") pod "global-pull-secret-syncer-986t7" (UID: "b9f5fe66-9376-47a2-b714-de84ffb90ad8") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:20.884472 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:20.884428 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:20.884672 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:20.884655 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:20.884736 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:20.884724 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs podName:4e47afd1-3679-47c0-bbdb-66ef0145e925 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:28.884708363 +0000 UTC m=+18.207131822 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs") pod "network-metrics-daemon-p6slt" (UID: "4e47afd1-3679-47c0-bbdb-66ef0145e925") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:20.985962 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:20.985916 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:20.986150 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:20.986124 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:56:20.986150 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:20.986145 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:56:20.986288 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:20.986159 2556 projected.go:196] Error preparing data for projected volume kube-api-access-4dkcn for pod openshift-network-diagnostics/network-check-target-94gw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:20.986288 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:20.986226 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn podName:deaef137-9bcb-4e3b-b07e-4bd32c57ba2e nodeName:}" failed. No retries permitted until 2026-08-04 15:56:28.986206335 +0000 UTC m=+18.308629846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dkcn" (UniqueName: "kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn") pod "network-check-target-94gw4" (UID: "deaef137-9bcb-4e3b-b07e-4bd32c57ba2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:21.271048 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:21.270490 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:21.271048 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:21.270496 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:21.271048 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:21.270631 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:21.271048 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:21.270626 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:21.271048 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:21.270776 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:21.271048 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:21.270864 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:23.268422 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:23.267911 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:23.268422 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:23.267921 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:23.268422 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:23.268051 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:23.268422 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:23.268139 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:23.268422 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:23.268133 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:23.268422 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:23.268231 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:25.267424 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:25.267386 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:25.267424 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:25.267427 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:25.267924 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:25.267386 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:25.267924 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:25.267527 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:25.267924 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:25.267600 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:25.267924 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:25.267682 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:27.267377 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:27.267339 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:27.267377 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:27.267373 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:27.267920 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:27.267339 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:27.267920 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:27.267458 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:27.267920 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:27.267511 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:27.267920 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:27.267591 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:28.847180 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:28.847141 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:28.847758 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:28.847283 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:28.847758 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:28.847354 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret podName:b9f5fe66-9376-47a2-b714-de84ffb90ad8 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:44.847332416 +0000 UTC m=+34.169755882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret") pod "global-pull-secret-syncer-986t7" (UID: "b9f5fe66-9376-47a2-b714-de84ffb90ad8") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:28.947901 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:28.947867 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:28.948080 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:28.948036 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:28.948155 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:28.948106 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs podName:4e47afd1-3679-47c0-bbdb-66ef0145e925 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:44.948085546 +0000 UTC m=+34.270509009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs") pod "network-metrics-daemon-p6slt" (UID: "4e47afd1-3679-47c0-bbdb-66ef0145e925") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:29.048545 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:29.048512 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:29.048719 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:29.048652 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:56:29.048719 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:29.048680 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:56:29.048719 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:29.048694 2556 projected.go:196] Error preparing data for projected volume kube-api-access-4dkcn for pod openshift-network-diagnostics/network-check-target-94gw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:29.048876 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:29.048777 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn podName:deaef137-9bcb-4e3b-b07e-4bd32c57ba2e nodeName:}" failed. No retries permitted until 2026-08-04 15:56:45.048757827 +0000 UTC m=+34.371181303 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dkcn" (UniqueName: "kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn") pod "network-check-target-94gw4" (UID: "deaef137-9bcb-4e3b-b07e-4bd32c57ba2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:29.267341 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:29.267263 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:29.267507 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:29.267264 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:29.267507 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:29.267412 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:29.267507 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:29.267273 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:29.267507 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:29.267480 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:29.267707 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:29.267534 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:31.268128 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:31.268095 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:31.268609 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:31.268207 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:31.268609 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:31.268216 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:31.268609 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:31.268231 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:31.268609 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:31.268304 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:31.268609 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:31.268386 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:33.267271 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:33.267223 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:33.267751 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:33.267351 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:33.267751 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:33.267370 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:33.267751 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:33.267389 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:33.267751 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:33.267478 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:33.267751 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:33.267557 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:34.414689 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.414438 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-scrxz" event={"ID":"de8b7d3f-9598-47f7-a387-8117c80acc3c","Type":"ContainerStarted","Data":"d0da87342bac8cc8b5658d1c8fca1b662b0c850028eaafdcf8bbeddb85fbbedf"} Aug 04 15:56:34.415869 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.415838 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ppxb" event={"ID":"6484c73f-9461-43ec-bf2a-924665d325a6","Type":"ContainerStarted","Data":"c72f3586dea36517630b00d548cd40fed2d458bf33b558639059b8e33145921d"} Aug 04 15:56:34.417265 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.417234 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vzlhw" event={"ID":"77cfdd06-8d08-4813-848f-43139a3c05b7","Type":"ContainerStarted","Data":"69442f629d0099c8bf2e600daa23e8b2428d7bc98af74201bf3af4aead89657e"} Aug 04 15:56:34.419400 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.419377 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" event={"ID":"f175e9b5-253c-420c-bebc-ce9fc78e82dc","Type":"ContainerStarted","Data":"41fcf79d2f6cc99c369be5eb8b9843c72f8d0e608375da1c255ac00dac30b38c"} Aug 04 15:56:34.419512 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.419436 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" event={"ID":"f175e9b5-253c-420c-bebc-ce9fc78e82dc","Type":"ContainerStarted","Data":"cddb480089066dca3f363f66bd4a360549f28706f366b2bfb492f37c4e8a2e40"} Aug 04 15:56:34.419573 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.419527 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" event={"ID":"f175e9b5-253c-420c-bebc-ce9fc78e82dc","Type":"ContainerStarted","Data":"bfb0e63985c0e42da53a75eb4ae7abb9ddc9724f97f34678f126ae85b4f10a46"} Aug 04 15:56:34.419573 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.419557 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" event={"ID":"f175e9b5-253c-420c-bebc-ce9fc78e82dc","Type":"ContainerStarted","Data":"81c40ad755c5f8b77b02e61c063dc91b9b5b71660bbe488fc044e9d3274abeb6"} Aug 04 15:56:34.420637 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.420612 2556 generic.go:356] "Generic (PLEG): container finished" podID="d05e8f95-eabc-4eb2-8f47-116955bc09a0" containerID="34c3dfdeb6489e0bd14333e847655691f3f11fbd83c504d44fe66c51b3ac0def" exitCode=0 Aug 04 15:56:34.420717 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.420696 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" event={"ID":"d05e8f95-eabc-4eb2-8f47-116955bc09a0","Type":"ContainerDied","Data":"34c3dfdeb6489e0bd14333e847655691f3f11fbd83c504d44fe66c51b3ac0def"} Aug 04 15:56:34.422132 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.422110 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62lcl" event={"ID":"ef8a69b0-b53d-4781-9265-5792c4aae2f0","Type":"ContainerStarted","Data":"ad14719dcb5ae1ccb85f48097d0c85b9c342a9f4f295937cd853a24bd13065ca"} Aug 04 15:56:34.423451 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.423420 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal" event={"ID":"ef5790e9892603695c64fc93d2f5fff7","Type":"ContainerStarted","Data":"63d9fc3ec257b785c0dddf128bdba3e77deeffa2f68daf94d374edb34c46eb24"} Aug 04 15:56:34.424842 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.424823 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" event={"ID":"3ad51ce2-face-404d-af19-4c0bfe22c59a","Type":"ContainerStarted","Data":"825d755877b34a6fdd702d9fde124954aff693bc6a3a25af43da46e3fe1eb864"} Aug 04 15:56:34.425978 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.425954 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nrkz8" event={"ID":"7ad081f0-d1a9-4890-b1e9-4784a2f71d2a","Type":"ContainerStarted","Data":"eaf5edb75fe2284ba3d37bc526fa038f53a9afaf8da6dd3e72269ece1afbf460"} Aug 04 15:56:34.431334 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.431295 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-scrxz" podStartSLOduration=2.010991021 podStartE2EDuration="22.431285474s" totalImagesPullingTime="20.420294453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:34.431046538 +0000 UTC m=+23.753470018" watchObservedRunningTime="2026-08-04 15:56:34.431285474 +0000 UTC m=+23.753708953" Aug 04 15:56:34.431524 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.431506 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-227.ec2.internal" podStartSLOduration=21.431501851 podStartE2EDuration="21.431501851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:16.407273085 +0000 UTC m=+5.729696571" watchObservedRunningTime="2026-08-04 15:56:34.431501851 +0000 UTC m=+23.753925330" Aug 04 15:56:34.460155 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.460117 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nrkz8" podStartSLOduration=9.801871559 podStartE2EDuration="22.460101053s" totalImagesPullingTime="12.658229494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:34.452531708 +0000 UTC m=+23.774955184" watchObservedRunningTime="2026-08-04 15:56:34.460101053 +0000 UTC m=+23.782524532" Aug 04 15:56:34.475301 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.475257 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vzlhw" podStartSLOduration=1.981279061 podStartE2EDuration="22.475242136s" totalImagesPullingTime="20.493963075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:34.475014936 +0000 UTC m=+23.797438436" watchObservedRunningTime="2026-08-04 15:56:34.475242136 +0000 UTC m=+23.797665616" Aug 04 15:56:34.490799 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.490760 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4ppxb" podStartSLOduration=2.069264322 podStartE2EDuration="22.490745898s" totalImagesPullingTime="20.421481576s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:34.490587655 +0000 UTC m=+23.813011135" watchObservedRunningTime="2026-08-04 15:56:34.490745898 +0000 UTC m=+23.813169376" Aug 04 15:56:34.508317 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.508279 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-227.ec2.internal" podStartSLOduration=21.508262351 podStartE2EDuration="21.508262351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:34.507823056 +0000 UTC m=+23.830246539" watchObservedRunningTime="2026-08-04 15:56:34.508262351 +0000 UTC m=+23.830685831" Aug 04 15:56:34.532531 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:34.531626 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-62lcl" podStartSLOduration=2.050307589 podStartE2EDuration="22.531607346s" totalImagesPullingTime="20.481299757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:34.530115917 +0000 UTC m=+23.852539423" watchObservedRunningTime="2026-08-04 15:56:34.531607346 +0000 UTC m=+23.854030831" Aug 04 15:56:35.271440 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:35.271409 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:35.271440 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:35.271439 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:35.271853 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:35.271444 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:35.271853 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:35.271535 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:35.271853 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:35.271680 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:35.271853 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:35.271782 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:35.430338 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:35.430304 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" event={"ID":"f175e9b5-253c-420c-bebc-ce9fc78e82dc","Type":"ContainerStarted","Data":"28b733827dcf199097cf43c0f7c6290bdd1802c90f18b762e1a5c5bfe608b1d4"} Aug 04 15:56:35.430710 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:35.430377 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" event={"ID":"f175e9b5-253c-420c-bebc-ce9fc78e82dc","Type":"ContainerStarted","Data":"d1cecf7c31009487a9de08818c9370d78be9a39b9d263cb66466359b69f09ef1"} Aug 04 15:56:35.431679 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:35.431631 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdx54" event={"ID":"1f56a917-750c-4649-82c3-1a23788d486a","Type":"ContainerStarted","Data":"592589e972bee1f5b6e0fac2a5a5bb290de5e5c28871c5b9152d324837dd752c"} Aug 04 15:56:35.447454 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:35.447408 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sdx54" podStartSLOduration=2.999225943 podStartE2EDuration="23.447393388s" totalImagesPullingTime="20.448167445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:35.447146754 +0000 UTC m=+24.769570234" watchObservedRunningTime="2026-08-04 15:56:35.447393388 +0000 UTC m=+24.769816867" Aug 04 15:56:35.458161 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:35.458134 2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 15:56:36.271385 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:36.271189 2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T15:56:35.458156265Z","UUID":"3d62cd00-0de4-472d-86e5-344042b75737","Handler":null,"Name":"","Endpoint":""} Aug 04 15:56:36.272995 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:36.272970 2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 15:56:36.273130 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:36.273012 2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 15:56:36.436089 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:36.436051 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" event={"ID":"3ad51ce2-face-404d-af19-4c0bfe22c59a","Type":"ContainerStarted","Data":"d11a832aa12a5824aec0a9904279e6ab55566724314d298f659c7125e3850443"} Aug 04 15:56:36.627145 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:36.627106 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:36.627887 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:36.627860 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:36.630473 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:36.630448 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:36.630911 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:36.630889 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nrkz8" Aug 04 15:56:37.266864 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:37.266831 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:37.266864 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:37.266855 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:37.267142 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:37.266978 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:37.267142 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:37.267118 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:37.267246 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:37.267209 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:37.267346 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:37.267327 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:37.441597 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:37.441561 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" event={"ID":"f175e9b5-253c-420c-bebc-ce9fc78e82dc","Type":"ContainerStarted","Data":"d506ec82adb795d6d9c493abec381f3703342a8f521d615494c1d89c1fc6ac20"} Aug 04 15:56:39.267266 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:39.267012 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:39.267838 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:39.267012 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:39.267838 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:39.267325 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:39.267838 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:39.267040 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:39.267838 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:39.267389 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:39.267838 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:39.267487 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:40.452133 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:40.451937 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" event={"ID":"f175e9b5-253c-420c-bebc-ce9fc78e82dc","Type":"ContainerStarted","Data":"e1660962287ce64e8931acd5fb6bd5126aa6f8127a35100c653e37f904e49ccc"} Aug 04 15:56:40.452862 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:40.452199 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:40.457848 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:40.457822 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" event={"ID":"3ad51ce2-face-404d-af19-4c0bfe22c59a","Type":"ContainerStarted","Data":"f70afd9373aad01b55391c742201aa53334dd78ce5c36365642984ed88b2936d"} Aug 04 15:56:40.468779 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:40.468755 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:40.494255 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:40.494208 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" podStartSLOduration=7.982634349 podStartE2EDuration="28.494193336s" totalImagesPullingTime="20.511558987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:40.492581064 +0000 UTC m=+29.815004544" watchObservedRunningTime="2026-08-04 15:56:40.494193336 +0000 UTC m=+29.816616817" Aug 04 15:56:40.515084 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:40.515035 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4gpq" podStartSLOduration=2.197519275 podStartE2EDuration="28.515018719s" totalImagesPullingTime="26.317499444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:40.51486825 +0000 UTC m=+29.837291729" watchObservedRunningTime="2026-08-04 15:56:40.515018719 +0000 UTC m=+29.837442199" Aug 04 15:56:41.268209 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:41.268176 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:41.268458 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:41.268244 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:41.268458 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:41.268253 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:41.268458 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:41.268361 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:41.268458 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:41.268401 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:41.268458 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:41.268440 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:41.462650 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:41.462551 2556 generic.go:356] "Generic (PLEG): container finished" podID="d05e8f95-eabc-4eb2-8f47-116955bc09a0" containerID="2ff2b174dacd7003c525a2c93c931d5ea2d6632b331a192403776188f49eba33" exitCode=0 Aug 04 15:56:41.462650 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:41.462639 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" event={"ID":"d05e8f95-eabc-4eb2-8f47-116955bc09a0","Type":"ContainerDied","Data":"2ff2b174dacd7003c525a2c93c931d5ea2d6632b331a192403776188f49eba33"} Aug 04 15:56:41.463937 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:41.463372 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:41.463937 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:41.463426 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:41.490062 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:41.490034 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:56:42.463124 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:42.463090 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p6slt"] Aug 04 15:56:42.463684 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:42.463262 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:42.463684 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:42.463373 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:42.465446 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:42.465420 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-94gw4"] Aug 04 15:56:42.465580 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:42.465539 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:42.465643 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:42.465629 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:42.479006 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:42.478977 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-986t7"] Aug 04 15:56:42.479180 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:42.479162 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:42.479301 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:42.479275 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:43.469743 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:43.469555 2556 generic.go:356] "Generic (PLEG): container finished" podID="d05e8f95-eabc-4eb2-8f47-116955bc09a0" containerID="0e30d21b25e9dd7279a2581fb34508dda9d6cace9d0bfb032f800c182e1588be" exitCode=0 Aug 04 15:56:43.470161 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:43.469631 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" event={"ID":"d05e8f95-eabc-4eb2-8f47-116955bc09a0","Type":"ContainerDied","Data":"0e30d21b25e9dd7279a2581fb34508dda9d6cace9d0bfb032f800c182e1588be"} Aug 04 15:56:44.266971 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:44.266918 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:44.266971 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:44.266964 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:44.267253 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:44.267040 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:44.267253 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:44.267053 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:44.267253 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:44.267144 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:44.267253 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:44.267219 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:44.862576 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:44.862552 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:44.862948 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:44.862663 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:44.862948 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:44.862709 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret podName:b9f5fe66-9376-47a2-b714-de84ffb90ad8 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:16.862696189 +0000 UTC m=+66.185119647 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret") pod "global-pull-secret-syncer-986t7" (UID: "b9f5fe66-9376-47a2-b714-de84ffb90ad8") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:56:44.963426 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:44.963340 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:44.963565 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:44.963489 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:44.963565 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:44.963555 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs podName:4e47afd1-3679-47c0-bbdb-66ef0145e925 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:16.963539899 +0000 UTC m=+66.285963357 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs") pod "network-metrics-daemon-p6slt" (UID: "4e47afd1-3679-47c0-bbdb-66ef0145e925") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:56:45.064446 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:45.064410 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:45.064595 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:45.064563 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:56:45.064595 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:45.064583 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:56:45.064595 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:45.064593 2556 projected.go:196] Error preparing data for projected volume kube-api-access-4dkcn for pod openshift-network-diagnostics/network-check-target-94gw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:45.064698 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:45.064644 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn podName:deaef137-9bcb-4e3b-b07e-4bd32c57ba2e nodeName:}" failed. No retries permitted until 2026-08-04 15:57:17.064628955 +0000 UTC m=+66.387052413 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dkcn" (UniqueName: "kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn") pod "network-check-target-94gw4" (UID: "deaef137-9bcb-4e3b-b07e-4bd32c57ba2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:56:45.476071 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:45.476035 2556 generic.go:356] "Generic (PLEG): container finished" podID="d05e8f95-eabc-4eb2-8f47-116955bc09a0" containerID="f8450c677bb5ed9cf4185cddd46310182fa100aa6f47d5c40e3e48ebadd0fe4f" exitCode=0 Aug 04 15:56:45.476244 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:45.476113 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" event={"ID":"d05e8f95-eabc-4eb2-8f47-116955bc09a0","Type":"ContainerDied","Data":"f8450c677bb5ed9cf4185cddd46310182fa100aa6f47d5c40e3e48ebadd0fe4f"} Aug 04 15:56:46.267315 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:46.267283 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:46.267850 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:46.267321 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:46.267850 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:46.267400 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:46.267850 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:46.267430 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94gw4" podUID="deaef137-9bcb-4e3b-b07e-4bd32c57ba2e" Aug 04 15:56:46.267850 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:46.267539 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p6slt" podUID="4e47afd1-3679-47c0-bbdb-66ef0145e925" Aug 04 15:56:46.267850 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:46.267611 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-986t7" podUID="b9f5fe66-9376-47a2-b714-de84ffb90ad8" Aug 04 15:56:47.016202 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.016167 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-227.ec2.internal" event="NodeReady" Aug 04 15:56:47.016383 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.016313 2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 15:56:47.060631 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.060598 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-757cb797ff-mn8b7"] Aug 04 15:56:47.064235 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.064212 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.069087 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.069057 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 04 15:56:47.069087 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.069061 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 04 15:56:47.069459 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.069439 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 04 15:56:47.069580 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.069440 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-l556n\"" Aug 04 15:56:47.080224 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.080192 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 04 15:56:47.085375 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.085341 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757cb797ff-mn8b7"] Aug 04 15:56:47.091797 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.091767 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ft67f"] Aug 04 15:56:47.096407 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.096379 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vmbgf"] Aug 04 15:56:47.096527 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.096516 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.099943 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.099920 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fnzc4\"" Aug 04 15:56:47.100063 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.099954 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 15:56:47.100063 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.099920 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 15:56:47.100433 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.100414 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vmbgf" Aug 04 15:56:47.102935 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.102915 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 15:56:47.103183 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.103165 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 15:56:47.103281 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.103192 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 15:56:47.103337 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.103316 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-752h4\"" Aug 04 15:56:47.106650 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.106627 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vmbgf"] Aug 04 15:56:47.111016 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.110990 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ft67f"] Aug 04 15:56:47.179953 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.179918 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-image-registry-private-configuration\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.179953 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.179958 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.180185 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.179980 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f92bfaa-cc35-49fb-af6c-79df9cbec888-ca-trust-extracted\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.180185 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180005 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-bound-sa-token\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.180185 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180060 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r94c5\" (UniqueName: \"kubernetes.io/projected/69637f37-b07c-498f-9fbc-35d463e51e58-kube-api-access-r94c5\") pod \"ingress-canary-vmbgf\" (UID: \"69637f37-b07c-498f-9fbc-35d463e51e58\") " pod="openshift-ingress-canary/ingress-canary-vmbgf" Aug 04 15:56:47.180185 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180086 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.180185 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180109 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/26589d8e-4d36-466c-993c-2d0fb23c2817-config-volume\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.180185 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180176 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-certificates\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.180461 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180221 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69637f37-b07c-498f-9fbc-35d463e51e58-cert\") pod \"ingress-canary-vmbgf\" (UID: \"69637f37-b07c-498f-9fbc-35d463e51e58\") " pod="openshift-ingress-canary/ingress-canary-vmbgf" Aug 04 15:56:47.180461 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180251 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-installation-pull-secrets\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.180461 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180293 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njl2n\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-kube-api-access-njl2n\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.180461 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180322 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26589d8e-4d36-466c-993c-2d0fb23c2817-tmp-dir\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.180461 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180349 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-trusted-ca\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.180461 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.180374 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6626\" (UniqueName: \"kubernetes.io/projected/26589d8e-4d36-466c-993c-2d0fb23c2817-kube-api-access-g6626\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.262017 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.261978 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nrkz8_7ad081f0-d1a9-4890-b1e9-4784a2f71d2a/konnectivity-agent/0.log" limit=1024 Aug 04 15:56:47.281023 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.280941 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69637f37-b07c-498f-9fbc-35d463e51e58-cert\") pod \"ingress-canary-vmbgf\" (UID: \"69637f37-b07c-498f-9fbc-35d463e51e58\") " pod="openshift-ingress-canary/ingress-canary-vmbgf" Aug 04 15:56:47.281023 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.280978 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-installation-pull-secrets\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281097 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njl2n\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-kube-api-access-njl2n\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281143 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26589d8e-4d36-466c-993c-2d0fb23c2817-tmp-dir\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281173 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-trusted-ca\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281200 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6626\" (UniqueName: \"kubernetes.io/projected/26589d8e-4d36-466c-993c-2d0fb23c2817-kube-api-access-g6626\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281268 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-image-registry-private-configuration\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281304 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281338 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f92bfaa-cc35-49fb-af6c-79df9cbec888-ca-trust-extracted\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281361 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-bound-sa-token\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281389 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r94c5\" (UniqueName: \"kubernetes.io/projected/69637f37-b07c-498f-9fbc-35d463e51e58-kube-api-access-r94c5\") pod \"ingress-canary-vmbgf\" (UID: \"69637f37-b07c-498f-9fbc-35d463e51e58\") " pod="openshift-ingress-canary/ingress-canary-vmbgf" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281417 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281444 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/26589d8e-4d36-466c-993c-2d0fb23c2817-config-volume\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.281561 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.281488 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-certificates\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.282118 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.281886 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:56:47.282118 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.281955 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls podName:26589d8e-4d36-466c-993c-2d0fb23c2817 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:47.781936288 +0000 UTC m=+37.104359761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls") pod "dns-default-ft67f" (UID: "26589d8e-4d36-466c-993c-2d0fb23c2817") : secret "dns-default-metrics-tls" not found Aug 04 15:56:47.282118 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.282100 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-certificates\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.282278 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.282106 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:56:47.282278 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.282143 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757cb797ff-mn8b7: secret "image-registry-tls" not found Aug 04 15:56:47.282278 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.282163 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f92bfaa-cc35-49fb-af6c-79df9cbec888-ca-trust-extracted\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.282278 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.282192 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls podName:0f92bfaa-cc35-49fb-af6c-79df9cbec888 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:47.782179169 +0000 UTC m=+37.104602641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls") pod "image-registry-757cb797ff-mn8b7" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888") : secret "image-registry-tls" not found Aug 04 15:56:47.282687 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.282659 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/26589d8e-4d36-466c-993c-2d0fb23c2817-config-volume\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.282820 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.282691 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26589d8e-4d36-466c-993c-2d0fb23c2817-tmp-dir\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.283159 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.283135 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-trusted-ca\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.286867 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.286841 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-installation-pull-secrets\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.286981 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.286845 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-image-registry-private-configuration\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.286981 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.286851 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69637f37-b07c-498f-9fbc-35d463e51e58-cert\") pod \"ingress-canary-vmbgf\" (UID: \"69637f37-b07c-498f-9fbc-35d463e51e58\") " pod="openshift-ingress-canary/ingress-canary-vmbgf" Aug 04 15:56:47.293990 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.293960 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r94c5\" (UniqueName: \"kubernetes.io/projected/69637f37-b07c-498f-9fbc-35d463e51e58-kube-api-access-r94c5\") pod \"ingress-canary-vmbgf\" (UID: \"69637f37-b07c-498f-9fbc-35d463e51e58\") " pod="openshift-ingress-canary/ingress-canary-vmbgf" Aug 04 15:56:47.294136 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.294017 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njl2n\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-kube-api-access-njl2n\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.294436 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.294414 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6626\" (UniqueName: \"kubernetes.io/projected/26589d8e-4d36-466c-993c-2d0fb23c2817-kube-api-access-g6626\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.294652 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.294632 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-bound-sa-token\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.369914 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.369877 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nrkz8_7ad081f0-d1a9-4890-b1e9-4784a2f71d2a/konnectivity-agent/0.log" limit=1024 Aug 04 15:56:47.415159 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.415120 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vmbgf" Aug 04 15:56:47.572208 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.571532 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vmbgf"] Aug 04 15:56:47.575450 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:56:47.575418 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69637f37_b07c_498f_9fbc_35d463e51e58.slice/crio-e177e1bb27ac721c78b0a90cd095ab8e0188ca671fc360151068d544acbd8689 WatchSource:0}: Error finding container e177e1bb27ac721c78b0a90cd095ab8e0188ca671fc360151068d544acbd8689: Status 404 returned error can't find the container with id e177e1bb27ac721c78b0a90cd095ab8e0188ca671fc360151068d544acbd8689 Aug 04 15:56:47.788463 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.788432 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:47.788642 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:47.788481 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:47.788642 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.788602 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:56:47.788642 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.788621 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757cb797ff-mn8b7: secret "image-registry-tls" not found Aug 04 15:56:47.788642 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.788599 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:56:47.788834 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.788692 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls podName:0f92bfaa-cc35-49fb-af6c-79df9cbec888 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:48.788666987 +0000 UTC m=+38.111090445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls") pod "image-registry-757cb797ff-mn8b7" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888") : secret "image-registry-tls" not found Aug 04 15:56:47.788834 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:47.788714 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls podName:26589d8e-4d36-466c-993c-2d0fb23c2817 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:48.788704977 +0000 UTC m=+38.111128443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls") pod "dns-default-ft67f" (UID: "26589d8e-4d36-466c-993c-2d0fb23c2817") : secret "dns-default-metrics-tls" not found Aug 04 15:56:48.266793 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.266760 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:56:48.266793 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.266780 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:56:48.267084 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.266760 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:56:48.271536 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.270960 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 15:56:48.271536 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.271187 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 15:56:48.271536 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.271394 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bfprq\"" Aug 04 15:56:48.271828 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.271577 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 15:56:48.271828 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.271767 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 15:56:48.271933 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.271890 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xj8nj\"" Aug 04 15:56:48.493555 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.493452 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vmbgf" event={"ID":"69637f37-b07c-498f-9fbc-35d463e51e58","Type":"ContainerStarted","Data":"e177e1bb27ac721c78b0a90cd095ab8e0188ca671fc360151068d544acbd8689"} Aug 04 15:56:48.795733 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.795670 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:48.795905 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:48.795745 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:48.795905 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:48.795851 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:56:48.795905 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:48.795863 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:56:48.795905 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:48.795881 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757cb797ff-mn8b7: secret "image-registry-tls" not found Aug 04 15:56:48.796117 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:48.795924 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls podName:26589d8e-4d36-466c-993c-2d0fb23c2817 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:50.795904366 +0000 UTC m=+40.118327833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls") pod "dns-default-ft67f" (UID: "26589d8e-4d36-466c-993c-2d0fb23c2817") : secret "dns-default-metrics-tls" not found Aug 04 15:56:48.796117 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:48.795939 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls podName:0f92bfaa-cc35-49fb-af6c-79df9cbec888 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:50.79593341 +0000 UTC m=+40.118356867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls") pod "image-registry-757cb797ff-mn8b7" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888") : secret "image-registry-tls" not found Aug 04 15:56:50.810485 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:50.810448 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:50.810916 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:50.810490 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:50.810916 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:50.810598 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:56:50.810916 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:50.810662 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls podName:26589d8e-4d36-466c-993c-2d0fb23c2817 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:54.810647752 +0000 UTC m=+44.133071214 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls") pod "dns-default-ft67f" (UID: "26589d8e-4d36-466c-993c-2d0fb23c2817") : secret "dns-default-metrics-tls" not found Aug 04 15:56:50.810916 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:50.810601 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:56:50.810916 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:50.810696 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757cb797ff-mn8b7: secret "image-registry-tls" not found Aug 04 15:56:50.810916 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:50.810753 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls podName:0f92bfaa-cc35-49fb-af6c-79df9cbec888 nodeName:}" failed. No retries permitted until 2026-08-04 15:56:54.810740834 +0000 UTC m=+44.133164307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls") pod "image-registry-757cb797ff-mn8b7" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888") : secret "image-registry-tls" not found Aug 04 15:56:51.497137 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:51.496968 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nrkz8_7ad081f0-d1a9-4890-b1e9-4784a2f71d2a/konnectivity-agent/0.log" limit=1024 Aug 04 15:56:51.502789 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:51.502766 2556 generic.go:356] "Generic (PLEG): container finished" podID="d05e8f95-eabc-4eb2-8f47-116955bc09a0" containerID="28590959ab485cc130d61995d514fa7677473ac5e036ddd5bdd0384607631b2e" exitCode=0 Aug 04 15:56:51.502925 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:51.502848 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" event={"ID":"d05e8f95-eabc-4eb2-8f47-116955bc09a0","Type":"ContainerDied","Data":"28590959ab485cc130d61995d514fa7677473ac5e036ddd5bdd0384607631b2e"} Aug 04 15:56:51.504247 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:51.504226 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vmbgf" event={"ID":"69637f37-b07c-498f-9fbc-35d463e51e58","Type":"ContainerStarted","Data":"fe3238b33f92c92225efc72f2278fd9471eb3115dcf064e44dc9c391c7b4ee4c"} Aug 04 15:56:52.509523 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:52.509486 2556 generic.go:356] "Generic (PLEG): container finished" podID="d05e8f95-eabc-4eb2-8f47-116955bc09a0" containerID="34b1f791ffa8c5b711b09120c864339dcaf31d05a6b50d0bc72e1b2eff77c2c8" exitCode=0 Aug 04 15:56:52.509964 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:52.509571 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" event={"ID":"d05e8f95-eabc-4eb2-8f47-116955bc09a0","Type":"ContainerDied","Data":"34b1f791ffa8c5b711b09120c864339dcaf31d05a6b50d0bc72e1b2eff77c2c8"} Aug 04 15:56:52.545090 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:52.545044 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vmbgf" podStartSLOduration=2.186333349 podStartE2EDuration="5.545001904s" totalImagesPullingTime="3.358668555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:51.567601249 +0000 UTC m=+40.890024733" watchObservedRunningTime="2026-08-04 15:56:52.545001904 +0000 UTC m=+41.867425383" Aug 04 15:56:53.165804 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:53.165771 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nrkz8_7ad081f0-d1a9-4890-b1e9-4784a2f71d2a/konnectivity-agent/0.log" limit=1024 Aug 04 15:56:53.515006 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:53.514970 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" event={"ID":"d05e8f95-eabc-4eb2-8f47-116955bc09a0","Type":"ContainerStarted","Data":"e5720a5514562b9f483987cdf28312cbf0aeff6c4587a04614160c37a027c095"} Aug 04 15:56:53.544462 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:53.544413 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s8xlk" podStartSLOduration=6.766663611 podStartE2EDuration="41.544393149s" totalImagesPullingTime="34.777729538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:53.54274018 +0000 UTC m=+42.865163656" watchObservedRunningTime="2026-08-04 15:56:53.544393149 +0000 UTC m=+42.866816631" Aug 04 15:56:53.673098 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:53.673061 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nrkz8_7ad081f0-d1a9-4890-b1e9-4784a2f71d2a/konnectivity-agent/0.log" limit=1024 Aug 04 15:56:53.773706 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:53.773617 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nrkz8_7ad081f0-d1a9-4890-b1e9-4784a2f71d2a/konnectivity-agent/0.log" limit=1024 Aug 04 15:56:54.843231 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:54.843186 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:56:54.843231 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:56:54.843236 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:56:54.843633 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:54.843345 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:56:54.843633 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:54.843360 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757cb797ff-mn8b7: secret "image-registry-tls" not found Aug 04 15:56:54.843633 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:54.843373 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:56:54.843633 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:54.843424 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls podName:0f92bfaa-cc35-49fb-af6c-79df9cbec888 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:02.843405253 +0000 UTC m=+52.165828713 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls") pod "image-registry-757cb797ff-mn8b7" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888") : secret "image-registry-tls" not found Aug 04 15:56:54.843633 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:56:54.843442 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls podName:26589d8e-4d36-466c-993c-2d0fb23c2817 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:02.843433675 +0000 UTC m=+52.165857136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls") pod "dns-default-ft67f" (UID: "26589d8e-4d36-466c-993c-2d0fb23c2817") : secret "dns-default-metrics-tls" not found Aug 04 15:57:02.895260 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:02.895213 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:57:02.895260 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:02.895265 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:57:02.895757 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:02.895348 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:57:02.895757 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:02.895417 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls podName:26589d8e-4d36-466c-993c-2d0fb23c2817 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:18.895402119 +0000 UTC m=+68.217825577 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls") pod "dns-default-ft67f" (UID: "26589d8e-4d36-466c-993c-2d0fb23c2817") : secret "dns-default-metrics-tls" not found Aug 04 15:57:02.895757 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:02.895354 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:57:02.895757 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:02.895451 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757cb797ff-mn8b7: secret "image-registry-tls" not found Aug 04 15:57:02.895757 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:02.895505 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls podName:0f92bfaa-cc35-49fb-af6c-79df9cbec888 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:18.895489662 +0000 UTC m=+68.217913123 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls") pod "image-registry-757cb797ff-mn8b7" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888") : secret "image-registry-tls" not found Aug 04 15:57:13.480081 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:13.480055 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrjbh" Aug 04 15:57:16.885543 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:16.885499 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:57:16.888518 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:16.888499 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 15:57:16.899497 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:16.899473 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f5fe66-9376-47a2-b714-de84ffb90ad8-original-pull-secret\") pod \"global-pull-secret-syncer-986t7\" (UID: \"b9f5fe66-9376-47a2-b714-de84ffb90ad8\") " pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:57:16.985884 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:16.985832 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:57:16.988710 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:16.988682 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 15:57:16.996875 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:16.996858 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Aug 04 15:57:16.996945 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:16.996917 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs podName:4e47afd1-3679-47c0-bbdb-66ef0145e925 nodeName:}" failed. No retries permitted until 2026-08-04 15:58:20.996900521 +0000 UTC m=+130.319323979 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs") pod "network-metrics-daemon-p6slt" (UID: "4e47afd1-3679-47c0-bbdb-66ef0145e925") : secret "metrics-daemon-secret" not found Aug 04 15:57:17.086938 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.086901 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:57:17.092939 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.092910 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 15:57:17.099668 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.099645 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 15:57:17.101765 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.101751 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-986t7" Aug 04 15:57:17.109824 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.109802 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dkcn\" (UniqueName: \"kubernetes.io/projected/deaef137-9bcb-4e3b-b07e-4bd32c57ba2e-kube-api-access-4dkcn\") pod \"network-check-target-94gw4\" (UID: \"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e\") " pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:57:17.222709 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.222666 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-986t7"] Aug 04 15:57:17.226967 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:57:17.226935 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f5fe66_9376_47a2_b714_de84ffb90ad8.slice/crio-f43f3d957286e7f66ab749a97a959bf67fc27bacf4ae02d04e03875044c5c82a WatchSource:0}: Error finding container f43f3d957286e7f66ab749a97a959bf67fc27bacf4ae02d04e03875044c5c82a: Status 404 returned error can't find the container with id f43f3d957286e7f66ab749a97a959bf67fc27bacf4ae02d04e03875044c5c82a Aug 04 15:57:17.384369 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.384339 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xj8nj\"" Aug 04 15:57:17.391844 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.391815 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:57:17.533783 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.533752 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-94gw4"] Aug 04 15:57:17.537121 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:57:17.537091 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeaef137_9bcb_4e3b_b07e_4bd32c57ba2e.slice/crio-cbcff6e4b35910c9e0a8c58b2eeeb5fc8976d26d7eac67c74c18b301138b2fe4 WatchSource:0}: Error finding container cbcff6e4b35910c9e0a8c58b2eeeb5fc8976d26d7eac67c74c18b301138b2fe4: Status 404 returned error can't find the container with id cbcff6e4b35910c9e0a8c58b2eeeb5fc8976d26d7eac67c74c18b301138b2fe4 Aug 04 15:57:17.569540 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.569500 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-94gw4" event={"ID":"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e","Type":"ContainerStarted","Data":"cbcff6e4b35910c9e0a8c58b2eeeb5fc8976d26d7eac67c74c18b301138b2fe4"} Aug 04 15:57:17.570539 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:17.570514 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-986t7" event={"ID":"b9f5fe66-9376-47a2-b714-de84ffb90ad8","Type":"ContainerStarted","Data":"f43f3d957286e7f66ab749a97a959bf67fc27bacf4ae02d04e03875044c5c82a"} Aug 04 15:57:18.901248 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:18.901213 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:57:18.901720 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:18.901307 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:57:18.901720 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:18.901398 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:57:18.901720 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:18.901417 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:57:18.901720 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:18.901422 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757cb797ff-mn8b7: secret "image-registry-tls" not found Aug 04 15:57:18.901720 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:18.901486 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls podName:26589d8e-4d36-466c-993c-2d0fb23c2817 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:50.901465765 +0000 UTC m=+100.223889224 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls") pod "dns-default-ft67f" (UID: "26589d8e-4d36-466c-993c-2d0fb23c2817") : secret "dns-default-metrics-tls" not found Aug 04 15:57:18.901720 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:18.901506 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls podName:0f92bfaa-cc35-49fb-af6c-79df9cbec888 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:50.901496661 +0000 UTC m=+100.223920123 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls") pod "image-registry-757cb797ff-mn8b7" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888") : secret "image-registry-tls" not found Aug 04 15:57:22.584271 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:22.584235 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-986t7" event={"ID":"b9f5fe66-9376-47a2-b714-de84ffb90ad8","Type":"ContainerStarted","Data":"5b67b8cc404f69a64077fd0065ff7b734b92e40facc6707f1d2066dc60c1a414"} Aug 04 15:57:22.610863 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:22.610814 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-986t7" podStartSLOduration=66.230345485 podStartE2EDuration="1m10.610798656s" totalImagesPullingTime="4.380453171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:57:22.609192894 +0000 UTC m=+71.931616374" watchObservedRunningTime="2026-08-04 15:57:22.610798656 +0000 UTC m=+71.933222135" Aug 04 15:57:24.877037 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.877006 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq"] Aug 04 15:57:24.906022 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.905995 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq"] Aug 04 15:57:24.906163 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.906084 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:24.910458 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.910436 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Aug 04 15:57:24.910458 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.910456 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Aug 04 15:57:24.910707 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.910694 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7mcdg\"" Aug 04 15:57:24.943320 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.943292 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:24.943470 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.943325 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:24.970470 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.968987 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g"] Aug 04 15:57:24.992115 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.992088 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g"] Aug 04 15:57:24.992256 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.992184 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:24.996346 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.996323 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Aug 04 15:57:24.996629 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.996612 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vvgbc\"" Aug 04 15:57:24.996858 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.996843 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:57:24.998262 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:24.998241 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Aug 04 15:57:25.043893 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.043869 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:25.044004 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.043903 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ntjt\" (UniqueName: \"kubernetes.io/projected/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-kube-api-access-7ntjt\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:25.044004 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.043926 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:25.044004 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.043965 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:25.044094 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:25.044050 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:57:25.044151 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:25.044138 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert podName:76f97e54-1257-4ba8-9033-3ab3b44b9eb8 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:25.544114843 +0000 UTC m=+74.866538309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-mgnfq" (UID: "76f97e54-1257-4ba8-9033-3ab3b44b9eb8") : secret "networking-console-plugin-cert" not found Aug 04 15:57:25.044739 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.044699 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:25.145361 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.145272 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:25.145489 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.145376 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ntjt\" (UniqueName: \"kubernetes.io/projected/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-kube-api-access-7ntjt\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:25.145489 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:25.145433 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:57:25.145563 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:25.145500 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls podName:b486eedc-c167-41c3-bc2b-1fcaf236ce1f nodeName:}" failed. No retries permitted until 2026-08-04 15:57:25.645482167 +0000 UTC m=+74.967905630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-smk8g" (UID: "b486eedc-c167-41c3-bc2b-1fcaf236ce1f") : secret "samples-operator-tls" not found Aug 04 15:57:25.154336 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.154306 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ntjt\" (UniqueName: \"kubernetes.io/projected/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-kube-api-access-7ntjt\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:25.548237 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.548195 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:25.548410 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:25.548345 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:57:25.548410 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:25.548400 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert podName:76f97e54-1257-4ba8-9033-3ab3b44b9eb8 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:26.54838687 +0000 UTC m=+75.870810327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-mgnfq" (UID: "76f97e54-1257-4ba8-9033-3ab3b44b9eb8") : secret "networking-console-plugin-cert" not found Aug 04 15:57:25.593585 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.593550 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-94gw4" event={"ID":"deaef137-9bcb-4e3b-b07e-4bd32c57ba2e","Type":"ContainerStarted","Data":"bd70a1b560dea140632f3f3679b088d2436d2b0fcd29d74361b4830aa1fe7968"} Aug 04 15:57:25.593752 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.593687 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:57:25.614079 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.614031 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-94gw4" podStartSLOduration=66.175498288 podStartE2EDuration="1m13.614015257s" totalImagesPullingTime="7.438516969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:57:25.613092921 +0000 UTC m=+74.935516401" watchObservedRunningTime="2026-08-04 15:57:25.614015257 +0000 UTC m=+74.936438737" Aug 04 15:57:25.648830 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:25.648800 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:25.648953 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:25.648920 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:57:25.648990 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:25.648971 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls podName:b486eedc-c167-41c3-bc2b-1fcaf236ce1f nodeName:}" failed. No retries permitted until 2026-08-04 15:57:26.64895785 +0000 UTC m=+75.971381309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-smk8g" (UID: "b486eedc-c167-41c3-bc2b-1fcaf236ce1f") : secret "samples-operator-tls" not found Aug 04 15:57:26.557098 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:26.557058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:26.557467 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:26.557207 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:57:26.557467 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:26.557273 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert podName:76f97e54-1257-4ba8-9033-3ab3b44b9eb8 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:28.557254876 +0000 UTC m=+77.879678340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-mgnfq" (UID: "76f97e54-1257-4ba8-9033-3ab3b44b9eb8") : secret "networking-console-plugin-cert" not found Aug 04 15:57:26.658117 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:26.658082 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:26.658285 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:26.658224 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:57:26.658285 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:26.658281 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls podName:b486eedc-c167-41c3-bc2b-1fcaf236ce1f nodeName:}" failed. No retries permitted until 2026-08-04 15:57:28.658263935 +0000 UTC m=+77.980687394 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-smk8g" (UID: "b486eedc-c167-41c3-bc2b-1fcaf236ce1f") : secret "samples-operator-tls" not found Aug 04 15:57:28.569919 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:28.569877 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:28.570288 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:28.570031 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:57:28.570288 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:28.570099 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert podName:76f97e54-1257-4ba8-9033-3ab3b44b9eb8 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:32.570083962 +0000 UTC m=+81.892507424 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-mgnfq" (UID: "76f97e54-1257-4ba8-9033-3ab3b44b9eb8") : secret "networking-console-plugin-cert" not found Aug 04 15:57:28.670956 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:28.670921 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:28.671132 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:28.671042 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:57:28.671132 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:28.671095 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls podName:b486eedc-c167-41c3-bc2b-1fcaf236ce1f nodeName:}" failed. No retries permitted until 2026-08-04 15:57:32.671081226 +0000 UTC m=+81.993504684 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-smk8g" (UID: "b486eedc-c167-41c3-bc2b-1fcaf236ce1f") : secret "samples-operator-tls" not found Aug 04 15:57:31.292904 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.292871 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-ftmvb"] Aug 04 15:57:31.295601 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.295584 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.298333 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.298309 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Aug 04 15:57:31.299424 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.299401 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Aug 04 15:57:31.299539 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.299439 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Aug 04 15:57:31.299539 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.299476 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qrtlq\"" Aug 04 15:57:31.299539 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.299497 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Aug 04 15:57:31.304467 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.304446 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-ftmvb"] Aug 04 15:57:31.387852 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.387819 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd571d9f-4cbc-4c05-8405-add76b8aaffb-signing-cabundle\") pod \"service-ca-98c88fbb5-ftmvb\" (UID: \"bd571d9f-4cbc-4c05-8405-add76b8aaffb\") " pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.388019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.387896 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s4wd\" (UniqueName: \"kubernetes.io/projected/bd571d9f-4cbc-4c05-8405-add76b8aaffb-kube-api-access-5s4wd\") pod \"service-ca-98c88fbb5-ftmvb\" (UID: \"bd571d9f-4cbc-4c05-8405-add76b8aaffb\") " pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.388019 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.387955 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd571d9f-4cbc-4c05-8405-add76b8aaffb-signing-key\") pod \"service-ca-98c88fbb5-ftmvb\" (UID: \"bd571d9f-4cbc-4c05-8405-add76b8aaffb\") " pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.488920 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.488887 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd571d9f-4cbc-4c05-8405-add76b8aaffb-signing-cabundle\") pod \"service-ca-98c88fbb5-ftmvb\" (UID: \"bd571d9f-4cbc-4c05-8405-add76b8aaffb\") " pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.489073 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.488934 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s4wd\" (UniqueName: \"kubernetes.io/projected/bd571d9f-4cbc-4c05-8405-add76b8aaffb-kube-api-access-5s4wd\") pod \"service-ca-98c88fbb5-ftmvb\" (UID: \"bd571d9f-4cbc-4c05-8405-add76b8aaffb\") " pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.489073 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.488974 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd571d9f-4cbc-4c05-8405-add76b8aaffb-signing-key\") pod \"service-ca-98c88fbb5-ftmvb\" (UID: \"bd571d9f-4cbc-4c05-8405-add76b8aaffb\") " pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.489650 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.489626 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd571d9f-4cbc-4c05-8405-add76b8aaffb-signing-cabundle\") pod \"service-ca-98c88fbb5-ftmvb\" (UID: \"bd571d9f-4cbc-4c05-8405-add76b8aaffb\") " pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.491260 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.491237 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd571d9f-4cbc-4c05-8405-add76b8aaffb-signing-key\") pod \"service-ca-98c88fbb5-ftmvb\" (UID: \"bd571d9f-4cbc-4c05-8405-add76b8aaffb\") " pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.498266 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.498245 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s4wd\" (UniqueName: \"kubernetes.io/projected/bd571d9f-4cbc-4c05-8405-add76b8aaffb-kube-api-access-5s4wd\") pod \"service-ca-98c88fbb5-ftmvb\" (UID: \"bd571d9f-4cbc-4c05-8405-add76b8aaffb\") " pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.605129 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.605028 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" Aug 04 15:57:31.729107 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:31.729075 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-ftmvb"] Aug 04 15:57:31.732014 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:57:31.731988 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd571d9f_4cbc_4c05_8405_add76b8aaffb.slice/crio-80bb54c8f362636b9afd1f92204f535739843fd075d720f8b44d5c18fca1e248 WatchSource:0}: Error finding container 80bb54c8f362636b9afd1f92204f535739843fd075d720f8b44d5c18fca1e248: Status 404 returned error can't find the container with id 80bb54c8f362636b9afd1f92204f535739843fd075d720f8b44d5c18fca1e248 Aug 04 15:57:32.598115 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:32.598078 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:32.598551 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:32.598250 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:57:32.598551 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:32.598380 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert podName:76f97e54-1257-4ba8-9033-3ab3b44b9eb8 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:40.598358253 +0000 UTC m=+89.920781714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-mgnfq" (UID: "76f97e54-1257-4ba8-9033-3ab3b44b9eb8") : secret "networking-console-plugin-cert" not found Aug 04 15:57:32.612423 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:32.612380 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" event={"ID":"bd571d9f-4cbc-4c05-8405-add76b8aaffb","Type":"ContainerStarted","Data":"80bb54c8f362636b9afd1f92204f535739843fd075d720f8b44d5c18fca1e248"} Aug 04 15:57:32.699254 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:32.699220 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:32.699423 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:32.699339 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:57:32.699423 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:32.699396 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls podName:b486eedc-c167-41c3-bc2b-1fcaf236ce1f nodeName:}" failed. No retries permitted until 2026-08-04 15:57:40.699377676 +0000 UTC m=+90.021801135 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-smk8g" (UID: "b486eedc-c167-41c3-bc2b-1fcaf236ce1f") : secret "samples-operator-tls" not found Aug 04 15:57:33.616309 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:33.616268 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" event={"ID":"bd571d9f-4cbc-4c05-8405-add76b8aaffb","Type":"ContainerStarted","Data":"d3c930ba466fb00bef7cb6a67a099fc8932e7dbf71f1c61291ed0be8ac85c4b8"} Aug 04 15:57:33.641680 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:33.641634 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-ftmvb" podStartSLOduration=0.917663741 podStartE2EDuration="2.641617992s" totalImagesPullingTime="1.723954251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:57:33.640226065 +0000 UTC m=+82.962649557" watchObservedRunningTime="2026-08-04 15:57:33.641617992 +0000 UTC m=+82.964041499" Aug 04 15:57:34.550970 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:34.550939 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-scrxz_de8b7d3f-9598-47f7-a387-8117c80acc3c/dns-node-resolver/0.log" Aug 04 15:57:35.550742 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:35.550699 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ppxb_6484c73f-9461-43ec-bf2a-924665d325a6/node-ca/0.log" Aug 04 15:57:36.750580 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:36.750551 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vmbgf_69637f37-b07c-498f-9fbc-35d463e51e58/serve-healthcheck-canary/0.log" Aug 04 15:57:40.660025 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:40.659986 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:40.660428 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:40.660123 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:57:40.660428 ip-10-0-132-227 kubenswrapper[2556]: E0804 15:57:40.660185 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert podName:76f97e54-1257-4ba8-9033-3ab3b44b9eb8 nodeName:}" failed. No retries permitted until 2026-08-04 15:57:56.660168769 +0000 UTC m=+105.982592231 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-mgnfq" (UID: "76f97e54-1257-4ba8-9033-3ab3b44b9eb8") : secret "networking-console-plugin-cert" not found Aug 04 15:57:40.761108 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:40.761073 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:40.763461 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:40.763434 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b486eedc-c167-41c3-bc2b-1fcaf236ce1f-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-smk8g\" (UID: \"b486eedc-c167-41c3-bc2b-1fcaf236ce1f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:40.900483 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:40.900439 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" Aug 04 15:57:41.033205 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:41.033034 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g"] Aug 04 15:57:41.638102 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:41.638065 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" event={"ID":"b486eedc-c167-41c3-bc2b-1fcaf236ce1f","Type":"ContainerStarted","Data":"75abd6649e1aebfd53cac5c89859ba075b7fb07124d94117cd69b1f9d8a89013"} Aug 04 15:57:43.645446 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:43.645395 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" event={"ID":"b486eedc-c167-41c3-bc2b-1fcaf236ce1f","Type":"ContainerStarted","Data":"60090021ed7f28f6628368fd2404f66c94e867ce63b106004d358aff233cbcbf"} Aug 04 15:57:43.645896 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:43.645460 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" event={"ID":"b486eedc-c167-41c3-bc2b-1fcaf236ce1f","Type":"ContainerStarted","Data":"5644e4601ce45bf114f620c1231d116c06c03dbc1267cfd7ca11b974144c128d"} Aug 04 15:57:43.662993 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:43.662937 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-smk8g" podStartSLOduration=17.908353557 podStartE2EDuration="19.662919063s" totalImagesPullingTime="1.754565506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:57:43.662807333 +0000 UTC m=+92.985230816" watchObservedRunningTime="2026-08-04 15:57:43.662919063 +0000 UTC m=+92.985342544" Aug 04 15:57:50.942125 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:50.942081 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:57:50.942125 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:50.942129 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:57:50.944490 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:50.944461 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26589d8e-4d36-466c-993c-2d0fb23c2817-metrics-tls\") pod \"dns-default-ft67f\" (UID: \"26589d8e-4d36-466c-993c-2d0fb23c2817\") " pod="openshift-dns/dns-default-ft67f" Aug 04 15:57:50.944616 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:50.944572 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"image-registry-757cb797ff-mn8b7\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:57:50.982123 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:50.982091 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-l556n\"" Aug 04 15:57:50.990380 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:50.990357 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:57:51.011261 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:51.011234 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fnzc4\"" Aug 04 15:57:51.019510 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:51.019483 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ft67f" Aug 04 15:57:51.137992 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:51.137960 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757cb797ff-mn8b7"] Aug 04 15:57:51.141008 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:57:51.140981 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f92bfaa_cc35_49fb_af6c_79df9cbec888.slice/crio-fc80c200e96a86f7cc324cb123add06a5bd7dd1773df2956ddc6379e04c2a9dd WatchSource:0}: Error finding container fc80c200e96a86f7cc324cb123add06a5bd7dd1773df2956ddc6379e04c2a9dd: Status 404 returned error can't find the container with id fc80c200e96a86f7cc324cb123add06a5bd7dd1773df2956ddc6379e04c2a9dd Aug 04 15:57:51.155941 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:51.155911 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ft67f"] Aug 04 15:57:51.158895 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:57:51.158865 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26589d8e_4d36_466c_993c_2d0fb23c2817.slice/crio-5622edf65c08bb1a92caaf211ae4457d598074aaeb28d202634aec13eabc33be WatchSource:0}: Error finding container 5622edf65c08bb1a92caaf211ae4457d598074aaeb28d202634aec13eabc33be: Status 404 returned error can't find the container with id 5622edf65c08bb1a92caaf211ae4457d598074aaeb28d202634aec13eabc33be Aug 04 15:57:51.670396 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:51.670356 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerStarted","Data":"a0b8422ec915c365118e73325284852408ac823af214d06a5c58a4c96783ddbc"} Aug 04 15:57:51.670590 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:51.670456 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerStarted","Data":"fc80c200e96a86f7cc324cb123add06a5bd7dd1773df2956ddc6379e04c2a9dd"} Aug 04 15:57:51.670590 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:51.670487 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:57:51.672144 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:51.672079 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ft67f" event={"ID":"26589d8e-4d36-466c-993c-2d0fb23c2817","Type":"ContainerStarted","Data":"5622edf65c08bb1a92caaf211ae4457d598074aaeb28d202634aec13eabc33be"} Aug 04 15:57:51.694315 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:51.694254 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podStartSLOduration=99.694233336 podStartE2EDuration="1m39.694233336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:57:51.692929972 +0000 UTC m=+101.015353453" watchObservedRunningTime="2026-08-04 15:57:51.694233336 +0000 UTC m=+101.016656818" Aug 04 15:57:53.682527 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:53.682436 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ft67f" event={"ID":"26589d8e-4d36-466c-993c-2d0fb23c2817","Type":"ContainerStarted","Data":"3c78321dff42a02310f93d571338e01feec5fae3340b7484d84557238f8d27b0"} Aug 04 15:57:53.682527 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:53.682489 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ft67f" event={"ID":"26589d8e-4d36-466c-993c-2d0fb23c2817","Type":"ContainerStarted","Data":"14ee00ab027a09605e5e309e33dd233e9f2eccbd95d24887ffc8c483ec9dd18d"} Aug 04 15:57:53.682941 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:53.682556 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ft67f" Aug 04 15:57:53.708441 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:53.708389 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ft67f" podStartSLOduration=64.864760335 podStartE2EDuration="1m6.708373473s" totalImagesPullingTime="1.843613138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:57:53.706851201 +0000 UTC m=+103.029274682" watchObservedRunningTime="2026-08-04 15:57:53.708373473 +0000 UTC m=+103.030796953" Aug 04 15:57:56.599222 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:56.599187 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-94gw4" Aug 04 15:57:56.686158 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:56.686120 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:56.688410 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:56.688384 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76f97e54-1257-4ba8-9033-3ab3b44b9eb8-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-mgnfq\" (UID: \"76f97e54-1257-4ba8-9033-3ab3b44b9eb8\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:56.715255 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:56.715232 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" Aug 04 15:57:56.839040 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:56.838893 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq"] Aug 04 15:57:56.841391 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:57:56.841362 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f97e54_1257_4ba8_9033_3ab3b44b9eb8.slice/crio-5df0dcdce65707a0ca2f1b4fcd51dabfbe33430e95550b7923aae85e6f5b6c94 WatchSource:0}: Error finding container 5df0dcdce65707a0ca2f1b4fcd51dabfbe33430e95550b7923aae85e6f5b6c94: Status 404 returned error can't find the container with id 5df0dcdce65707a0ca2f1b4fcd51dabfbe33430e95550b7923aae85e6f5b6c94 Aug 04 15:57:57.698616 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:57.698575 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" event={"ID":"76f97e54-1257-4ba8-9033-3ab3b44b9eb8","Type":"ContainerStarted","Data":"5df0dcdce65707a0ca2f1b4fcd51dabfbe33430e95550b7923aae85e6f5b6c94"} Aug 04 15:57:58.702775 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:58.702742 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" event={"ID":"76f97e54-1257-4ba8-9033-3ab3b44b9eb8","Type":"ContainerStarted","Data":"28042d7769d82b88f5e630ffb939f0d4754c1d68214550023e168e51ea13e2c3"} Aug 04 15:57:58.722291 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:57:58.722231 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-mgnfq" podStartSLOduration=33.651387146 podStartE2EDuration="34.722210695s" totalImagesPullingTime="1.070823549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:57:58.721387449 +0000 UTC m=+108.043810942" watchObservedRunningTime="2026-08-04 15:57:58.722210695 +0000 UTC m=+108.044634176" Aug 04 15:58:03.688811 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:03.688776 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ft67f" Aug 04 15:58:10.995495 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:10.995458 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:58:10.995967 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:10.995523 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:58:12.680981 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:12.680947 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:58:12.681351 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:12.681000 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:58:20.994340 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:20.994305 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:58:20.994709 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:20.994357 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:58:21.054801 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:21.054742 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:58:21.057084 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:21.057049 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e47afd1-3679-47c0-bbdb-66ef0145e925-metrics-certs\") pod \"network-metrics-daemon-p6slt\" (UID: \"4e47afd1-3679-47c0-bbdb-66ef0145e925\") " pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:58:21.297972 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:21.297892 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bfprq\"" Aug 04 15:58:21.306085 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:21.306063 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p6slt" Aug 04 15:58:21.444076 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:21.444046 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p6slt"] Aug 04 15:58:21.447329 ip-10-0-132-227 kubenswrapper[2556]: W0804 15:58:21.447296 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e47afd1_3679_47c0_bbdb_66ef0145e925.slice/crio-70cc6a0cb3984565b57c3f4cd4c7da49c9eac68b7ddd8dc35bed3002a1f3bfb6 WatchSource:0}: Error finding container 70cc6a0cb3984565b57c3f4cd4c7da49c9eac68b7ddd8dc35bed3002a1f3bfb6: Status 404 returned error can't find the container with id 70cc6a0cb3984565b57c3f4cd4c7da49c9eac68b7ddd8dc35bed3002a1f3bfb6 Aug 04 15:58:21.771282 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:21.771244 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p6slt" event={"ID":"4e47afd1-3679-47c0-bbdb-66ef0145e925","Type":"ContainerStarted","Data":"70cc6a0cb3984565b57c3f4cd4c7da49c9eac68b7ddd8dc35bed3002a1f3bfb6"} Aug 04 15:58:22.681398 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:22.681356 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:58:22.681785 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:22.681415 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:58:23.779812 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:23.779711 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p6slt" event={"ID":"4e47afd1-3679-47c0-bbdb-66ef0145e925","Type":"ContainerStarted","Data":"b0ef9b9f731e11d7cc0e01948fe0ba20d134ec3e31fc00adb3fb668907400dfa"} Aug 04 15:58:23.779812 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:23.779781 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p6slt" event={"ID":"4e47afd1-3679-47c0-bbdb-66ef0145e925","Type":"ContainerStarted","Data":"928946289f53fb404d158474bcb0bce83fc92eee495e9261f7cd0e08f43ca51f"} Aug 04 15:58:23.799015 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:23.798962 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p6slt" podStartSLOduration=129.728543403 podStartE2EDuration="2m11.798947049s" totalImagesPullingTime="2.070403646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:58:23.798534546 +0000 UTC m=+133.120958026" watchObservedRunningTime="2026-08-04 15:58:23.798947049 +0000 UTC m=+133.121370529" Aug 04 15:58:30.995309 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:30.995273 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:58:30.995680 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:30.995333 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:58:30.995680 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:30.995377 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:58:30.995943 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:30.995910 2556 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"a0b8422ec915c365118e73325284852408ac823af214d06a5c58a4c96783ddbc"} pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" containerMessage="Container registry failed liveness probe, will be restarted" Aug 04 15:58:31.000589 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:31.000555 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:58:31.000749 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:31.000603 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:58:40.999539 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:40.999503 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:58:40.999957 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:40.999554 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:58:44.086875 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:44.086843 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-mgnfq_76f97e54-1257-4ba8-9033-3ab3b44b9eb8/networking-console-plugin/0.log" Aug 04 15:58:44.686555 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:44.686519 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ft67f_26589d8e-4d36-466c-993c-2d0fb23c2817/dns/0.log" Aug 04 15:58:44.886641 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:44.886611 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ft67f_26589d8e-4d36-466c-993c-2d0fb23c2817/kube-rbac-proxy/0.log" Aug 04 15:58:45.886610 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:45.886581 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-scrxz_de8b7d3f-9598-47f7-a387-8117c80acc3c/dns-node-resolver/0.log" Aug 04 15:58:46.488335 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:46.488307 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-757cb797ff-mn8b7_0f92bfaa-cc35-49fb-af6c-79df9cbec888/registry/0.log" Aug 04 15:58:46.887380 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:46.887354 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ppxb_6484c73f-9461-43ec-bf2a-924665d325a6/node-ca/0.log" Aug 04 15:58:48.087491 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:48.087456 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vmbgf_69637f37-b07c-498f-9fbc-35d463e51e58/serve-healthcheck-canary/0.log" Aug 04 15:58:50.999656 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:50.999611 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:58:51.000058 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:50.999680 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:58:56.014552 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:56.014514 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" containerID="cri-o://a0b8422ec915c365118e73325284852408ac823af214d06a5c58a4c96783ddbc" gracePeriod=30 Aug 04 15:58:56.886330 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:56.886295 2556 generic.go:356] "Generic (PLEG): container finished" podID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerID="a0b8422ec915c365118e73325284852408ac823af214d06a5c58a4c96783ddbc" exitCode=0 Aug 04 15:58:56.886498 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:56.886365 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerDied","Data":"a0b8422ec915c365118e73325284852408ac823af214d06a5c58a4c96783ddbc"} Aug 04 15:58:56.886498 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:56.886438 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerStarted","Data":"0f7fec9f248ff7cd67080d7c1b6a3ea8ebcf8e8465452d83a0e65eff49390f69"} Aug 04 15:58:56.886624 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:58:56.886598 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:59:10.995243 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:10.995168 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:59:10.995243 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:10.995220 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:59:17.894476 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:17.894436 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:59:17.894947 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:17.894494 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:59:20.995364 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:20.995330 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:59:20.995758 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:20.995383 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:59:27.894318 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:27.894282 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:59:27.894706 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:27.894335 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:59:30.994585 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:30.994549 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:59:30.995017 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:30.994606 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:59:30.995017 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:30.994645 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 15:59:30.995129 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:30.995108 2556 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"0f7fec9f248ff7cd67080d7c1b6a3ea8ebcf8e8465452d83a0e65eff49390f69"} pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" containerMessage="Container registry failed liveness probe, will be restarted" Aug 04 15:59:30.998657 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:30.998628 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:59:30.998836 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:30.998677 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:59:41.000490 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:41.000448 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:59:41.000973 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:41.000508 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:59:50.998989 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:50.998944 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 15:59:50.999532 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:50.999016 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:59:56.012823 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:56.012772 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" containerID="cri-o://0f7fec9f248ff7cd67080d7c1b6a3ea8ebcf8e8465452d83a0e65eff49390f69" gracePeriod=30 Aug 04 15:59:57.078850 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:57.078819 2556 generic.go:356] "Generic (PLEG): container finished" podID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerID="0f7fec9f248ff7cd67080d7c1b6a3ea8ebcf8e8465452d83a0e65eff49390f69" exitCode=0 Aug 04 15:59:57.079199 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:57.078896 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerDied","Data":"0f7fec9f248ff7cd67080d7c1b6a3ea8ebcf8e8465452d83a0e65eff49390f69"} Aug 04 15:59:57.079199 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:57.078982 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerStarted","Data":"25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc"} Aug 04 15:59:57.079199 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:57.079006 2556 scope.go:117] "RemoveContainer" containerID="a0b8422ec915c365118e73325284852408ac823af214d06a5c58a4c96783ddbc" Aug 04 15:59:57.079199 ip-10-0-132-227 kubenswrapper[2556]: I0804 15:59:57.079115 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 16:00:10.995923 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:10.995882 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 16:00:10.996284 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:10.995938 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:00:18.088529 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:18.088493 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 16:00:18.088939 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:18.088558 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:00:20.994980 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:20.994929 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 16:00:20.995543 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:20.995466 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:00:28.088665 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:28.088626 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 16:00:28.089224 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:28.088686 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:00:30.995212 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:30.995179 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 16:00:30.995644 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:30.995240 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:00:30.995644 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:30.995277 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 16:00:30.995796 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:30.995685 2556 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc"} pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" containerMessage="Container registry failed liveness probe, will be restarted" Aug 04 16:00:30.999257 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:30.999227 2556 patch_prober.go:28] interesting pod/image-registry-757cb797ff-mn8b7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 16:00:30.999380 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:30.999282 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:00:31.600385 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.600353 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs"] Aug 04 16:00:31.603251 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.603232 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" Aug 04 16:00:31.608561 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.608511 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7w9h7\"" Aug 04 16:00:31.608561 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.608520 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Aug 04 16:00:31.608848 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.608609 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 04 16:00:31.608848 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.608625 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 04 16:00:31.612646 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.612628 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 04 16:00:31.618539 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.618516 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs"] Aug 04 16:00:31.716102 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.716054 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-h82zl"] Aug 04 16:00:31.719046 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.719028 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-h82zl" Aug 04 16:00:31.721326 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.721295 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b98d79c-b694-4b12-bcea-0b345531606b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dc959d48b-whncs\" (UID: \"4b98d79c-b694-4b12-bcea-0b345531606b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" Aug 04 16:00:31.721457 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.721359 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5nj\" (UniqueName: \"kubernetes.io/projected/4b98d79c-b694-4b12-bcea-0b345531606b-kube-api-access-cn5nj\") pod \"managed-serviceaccount-addon-agent-5dc959d48b-whncs\" (UID: \"4b98d79c-b694-4b12-bcea-0b345531606b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" Aug 04 16:00:31.724676 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.724655 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 04 16:00:31.724818 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.724783 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n77ng\"" Aug 04 16:00:31.724818 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.724802 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 04 16:00:31.743719 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.743692 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-h82zl"] Aug 04 16:00:31.794893 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.794866 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4njrx"] Aug 04 16:00:31.797931 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.797900 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:31.804825 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.804797 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 16:00:31.805149 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.805117 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 16:00:31.805243 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.805200 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dngfj\"" Aug 04 16:00:31.805303 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.805238 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 16:00:31.806367 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.806350 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 16:00:31.822452 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.822421 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5nj\" (UniqueName: \"kubernetes.io/projected/4b98d79c-b694-4b12-bcea-0b345531606b-kube-api-access-cn5nj\") pod \"managed-serviceaccount-addon-agent-5dc959d48b-whncs\" (UID: \"4b98d79c-b694-4b12-bcea-0b345531606b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" Aug 04 16:00:31.822594 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.822459 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b98d79c-b694-4b12-bcea-0b345531606b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dc959d48b-whncs\" (UID: \"4b98d79c-b694-4b12-bcea-0b345531606b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" Aug 04 16:00:31.822594 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.822480 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk6mx\" (UniqueName: \"kubernetes.io/projected/0e52402b-8a15-412e-87a2-598b4984aae8-kube-api-access-lk6mx\") pod \"downloads-77c974ffcf-h82zl\" (UID: \"0e52402b-8a15-412e-87a2-598b4984aae8\") " pod="openshift-console/downloads-77c974ffcf-h82zl" Aug 04 16:00:31.825090 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.825067 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b98d79c-b694-4b12-bcea-0b345531606b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dc959d48b-whncs\" (UID: \"4b98d79c-b694-4b12-bcea-0b345531606b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" Aug 04 16:00:31.851635 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.851553 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4njrx"] Aug 04 16:00:31.868191 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.868163 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5nj\" (UniqueName: \"kubernetes.io/projected/4b98d79c-b694-4b12-bcea-0b345531606b-kube-api-access-cn5nj\") pod \"managed-serviceaccount-addon-agent-5dc959d48b-whncs\" (UID: \"4b98d79c-b694-4b12-bcea-0b345531606b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" Aug 04 16:00:31.921013 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.920984 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" Aug 04 16:00:31.923504 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.923478 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk6mx\" (UniqueName: \"kubernetes.io/projected/0e52402b-8a15-412e-87a2-598b4984aae8-kube-api-access-lk6mx\") pod \"downloads-77c974ffcf-h82zl\" (UID: \"0e52402b-8a15-412e-87a2-598b4984aae8\") " pod="openshift-console/downloads-77c974ffcf-h82zl" Aug 04 16:00:31.923634 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.923517 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-crio-socket\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:31.923634 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.923540 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:31.923634 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.923556 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wrvk\" (UniqueName: \"kubernetes.io/projected/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-kube-api-access-9wrvk\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:31.923634 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.923597 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-data-volume\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:31.923864 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.923689 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:31.949978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:31.949950 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk6mx\" (UniqueName: \"kubernetes.io/projected/0e52402b-8a15-412e-87a2-598b4984aae8-kube-api-access-lk6mx\") pod \"downloads-77c974ffcf-h82zl\" (UID: \"0e52402b-8a15-412e-87a2-598b4984aae8\") " pod="openshift-console/downloads-77c974ffcf-h82zl" Aug 04 16:00:32.024039 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.024006 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-crio-socket\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.024432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.024056 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.024432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.024079 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wrvk\" (UniqueName: \"kubernetes.io/projected/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-kube-api-access-9wrvk\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.024432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.024103 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-data-volume\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.024432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.024141 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.024432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.024154 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-crio-socket\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.029294 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.024603 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-data-volume\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.029294 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.025074 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.029639 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.029614 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-h82zl" Aug 04 16:00:32.030058 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.030038 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.067887 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.067821 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wrvk\" (UniqueName: \"kubernetes.io/projected/e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b-kube-api-access-9wrvk\") pod \"insights-runtime-extractor-4njrx\" (UID: \"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b\") " pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.107577 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.107488 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4njrx" Aug 04 16:00:32.122247 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.122195 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs"] Aug 04 16:00:32.127803 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:00:32.127704 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b98d79c_b694_4b12_bcea_0b345531606b.slice/crio-bfa347101af4dc87c75a8a5920d4858cb2d2cedf4799ea8e63deee86cf333d33 WatchSource:0}: Error finding container bfa347101af4dc87c75a8a5920d4858cb2d2cedf4799ea8e63deee86cf333d33: Status 404 returned error can't find the container with id bfa347101af4dc87c75a8a5920d4858cb2d2cedf4799ea8e63deee86cf333d33 Aug 04 16:00:32.186925 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.186885 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" event={"ID":"4b98d79c-b694-4b12-bcea-0b345531606b","Type":"ContainerStarted","Data":"bfa347101af4dc87c75a8a5920d4858cb2d2cedf4799ea8e63deee86cf333d33"} Aug 04 16:00:32.219516 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.219488 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-h82zl"] Aug 04 16:00:32.244782 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:00:32.244750 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e52402b_8a15_412e_87a2_598b4984aae8.slice/crio-220a9b04abfa56de2dc8e9a4cfc46871a7597f4391498d85ef178a773a4df6e6 WatchSource:0}: Error finding container 220a9b04abfa56de2dc8e9a4cfc46871a7597f4391498d85ef178a773a4df6e6: Status 404 returned error can't find the container with id 220a9b04abfa56de2dc8e9a4cfc46871a7597f4391498d85ef178a773a4df6e6 Aug 04 16:00:32.283941 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:32.283898 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4njrx"] Aug 04 16:00:32.289279 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:00:32.289119 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51c47b6_5f45_40a7_bfe0_1f2bca6e2a2b.slice/crio-6c24cb14d721a0ba4e6bbc266e6474548ee93fcd27a984dc411476db9536f4da WatchSource:0}: Error finding container 6c24cb14d721a0ba4e6bbc266e6474548ee93fcd27a984dc411476db9536f4da: Status 404 returned error can't find the container with id 6c24cb14d721a0ba4e6bbc266e6474548ee93fcd27a984dc411476db9536f4da Aug 04 16:00:33.191788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:33.191755 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4njrx" event={"ID":"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b","Type":"ContainerStarted","Data":"f6f2e7f1f0a510ef89c48d1d955b793d4884342e4852a86a4ff1748f4eec1128"} Aug 04 16:00:33.192192 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:33.191811 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4njrx" event={"ID":"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b","Type":"ContainerStarted","Data":"6c24cb14d721a0ba4e6bbc266e6474548ee93fcd27a984dc411476db9536f4da"} Aug 04 16:00:33.192776 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:33.192749 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-h82zl" event={"ID":"0e52402b-8a15-412e-87a2-598b4984aae8","Type":"ContainerStarted","Data":"220a9b04abfa56de2dc8e9a4cfc46871a7597f4391498d85ef178a773a4df6e6"} Aug 04 16:00:34.607602 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.607569 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-lrrmg"] Aug 04 16:00:34.610594 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.610569 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.617494 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.617469 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 16:00:34.617494 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.617486 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 16:00:34.617716 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.617499 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Aug 04 16:00:34.617716 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.617558 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 16:00:34.617716 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.617470 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Aug 04 16:00:34.617716 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.617660 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hvsm7\"" Aug 04 16:00:34.634225 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.634196 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-lrrmg"] Aug 04 16:00:34.744957 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.744907 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.745166 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.744971 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-722vr\" (UniqueName: \"kubernetes.io/projected/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-kube-api-access-722vr\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.745166 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.745085 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.745166 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.745121 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.845854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.845815 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.846039 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.845858 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.846039 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.845898 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.846039 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.845932 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-722vr\" (UniqueName: \"kubernetes.io/projected/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-kube-api-access-722vr\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.846929 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.846900 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.848435 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.848410 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.848580 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.848561 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.858904 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.858837 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-722vr\" (UniqueName: \"kubernetes.io/projected/fe0db590-0b17-4aaa-b9da-df4837a4a0f0-kube-api-access-722vr\") pod \"prometheus-operator-758d7cb874-lrrmg\" (UID: \"fe0db590-0b17-4aaa-b9da-df4837a4a0f0\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:34.920647 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:34.920611 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" Aug 04 16:00:35.076341 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:35.076162 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-lrrmg"] Aug 04 16:00:35.078218 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:00:35.078192 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe0db590_0b17_4aaa_b9da_df4837a4a0f0.slice/crio-4c7f4bc4e637c7432c8410846f6601952c0a0951a82744f18d20a5d2918a35d1 WatchSource:0}: Error finding container 4c7f4bc4e637c7432c8410846f6601952c0a0951a82744f18d20a5d2918a35d1: Status 404 returned error can't find the container with id 4c7f4bc4e637c7432c8410846f6601952c0a0951a82744f18d20a5d2918a35d1 Aug 04 16:00:35.201683 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:35.201646 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4njrx" event={"ID":"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b","Type":"ContainerStarted","Data":"b793b97c18c61f5daa7966f9b95bfe20c67939a94b91084ae4b7ac29aceeea8b"} Aug 04 16:00:35.202712 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:35.202690 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" event={"ID":"fe0db590-0b17-4aaa-b9da-df4837a4a0f0","Type":"ContainerStarted","Data":"4c7f4bc4e637c7432c8410846f6601952c0a0951a82744f18d20a5d2918a35d1"} Aug 04 16:00:37.213595 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:37.213495 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" event={"ID":"fe0db590-0b17-4aaa-b9da-df4837a4a0f0","Type":"ContainerStarted","Data":"ee2f668dddd8fda4b84f2d70e853e709312762fa640c018b59960589531c5128"} Aug 04 16:00:37.214097 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:37.213613 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" event={"ID":"fe0db590-0b17-4aaa-b9da-df4837a4a0f0","Type":"ContainerStarted","Data":"e04f6bd792760418ffa8022031b30cdd253d66040f2820df0fa6f578041ba3cf"} Aug 04 16:00:37.238540 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:37.238216 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-lrrmg" podStartSLOduration=2.011889023 podStartE2EDuration="3.238198788s" totalImagesPullingTime="1.226309765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:00:37.237040366 +0000 UTC m=+266.559463845" watchObservedRunningTime="2026-08-04 16:00:37.238198788 +0000 UTC m=+266.560622267" Aug 04 16:00:39.086857 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.085197 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb"] Aug 04 16:00:39.089932 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.088907 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.091135 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.091108 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ztb4x"] Aug 04 16:00:39.093833 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.093134 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Aug 04 16:00:39.093833 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.093345 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Aug 04 16:00:39.093833 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.093551 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jdccv\"" Aug 04 16:00:39.095819 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.094416 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.096235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.096214 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Aug 04 16:00:39.097959 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.097108 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 16:00:39.097959 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.097419 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 16:00:39.097959 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.097604 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9rss6\"" Aug 04 16:00:39.097959 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.097827 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 16:00:39.111035 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.110934 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb"] Aug 04 16:00:39.188066 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188029 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.188260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188080 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.188260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188107 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpwwr\" (UniqueName: \"kubernetes.io/projected/3443e0ca-fb65-47ef-86ba-c3bc535fac61-kube-api-access-qpwwr\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.188260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188142 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/259aed9c-29fa-4a34-a9e6-8c1466819bb2-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.188260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188166 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3443e0ca-fb65-47ef-86ba-c3bc535fac61-root\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.188260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188202 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.188260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188228 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3443e0ca-fb65-47ef-86ba-c3bc535fac61-sys\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.188260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188247 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-wtmp\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.188605 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188278 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nssqz\" (UniqueName: \"kubernetes.io/projected/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-api-access-nssqz\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.188605 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188324 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-textfile\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.188605 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188352 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-accelerators-collector-config\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.188605 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188386 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/259aed9c-29fa-4a34-a9e6-8c1466819bb2-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.188605 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188411 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.188605 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188434 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-tls\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.188605 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.188460 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3443e0ca-fb65-47ef-86ba-c3bc535fac61-metrics-client-ca\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.225624 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.224911 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4njrx" event={"ID":"e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b","Type":"ContainerStarted","Data":"8b7444834a0ac407e80cdf923b7299160da235a5e1e32e2bdb144505036b7376"} Aug 04 16:00:39.227381 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.227350 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" event={"ID":"4b98d79c-b694-4b12-bcea-0b345531606b","Type":"ContainerStarted","Data":"c3198b8fc8aa13419bbb0431018b00eb0e067c572912f25f878081e7bbc9a81a"} Aug 04 16:00:39.252190 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.252113 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4njrx" podStartSLOduration=1.943368005 podStartE2EDuration="8.252090072s" totalImagesPullingTime="6.308722067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 16:00:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:00:39.250918911 +0000 UTC m=+268.573342393" watchObservedRunningTime="2026-08-04 16:00:39.252090072 +0000 UTC m=+268.574513534" Aug 04 16:00:39.282491 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.282424 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc959d48b-whncs" podStartSLOduration=1.692322314 podStartE2EDuration="8.282401871s" totalImagesPullingTime="6.590079557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:00:39.282360112 +0000 UTC m=+268.604783594" watchObservedRunningTime="2026-08-04 16:00:39.282401871 +0000 UTC m=+268.604825352" Aug 04 16:00:39.289825 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.289795 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.289988 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.289836 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.289988 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.289863 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpwwr\" (UniqueName: \"kubernetes.io/projected/3443e0ca-fb65-47ef-86ba-c3bc535fac61-kube-api-access-qpwwr\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.289988 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:00:39.289979 2556 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Aug 04 16:00:39.290142 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:00:39.290047 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-tls podName:259aed9c-29fa-4a34-a9e6-8c1466819bb2 nodeName:}" failed. No retries permitted until 2026-08-04 16:00:39.790027255 +0000 UTC m=+269.112450715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-kgblb" (UID: "259aed9c-29fa-4a34-a9e6-8c1466819bb2") : secret "kube-state-metrics-tls" not found Aug 04 16:00:39.290142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290076 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/259aed9c-29fa-4a34-a9e6-8c1466819bb2-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.290142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290119 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3443e0ca-fb65-47ef-86ba-c3bc535fac61-root\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.290299 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290166 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.290299 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290196 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3443e0ca-fb65-47ef-86ba-c3bc535fac61-sys\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.290299 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290219 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-wtmp\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.290299 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290251 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nssqz\" (UniqueName: \"kubernetes.io/projected/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-api-access-nssqz\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.290299 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290293 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-textfile\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.290935 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290320 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-accelerators-collector-config\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.290935 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290333 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3443e0ca-fb65-47ef-86ba-c3bc535fac61-sys\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.290935 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290353 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/259aed9c-29fa-4a34-a9e6-8c1466819bb2-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.290935 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290372 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3443e0ca-fb65-47ef-86ba-c3bc535fac61-root\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.290935 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290379 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.290935 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290429 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-tls\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.290935 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290455 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3443e0ca-fb65-47ef-86ba-c3bc535fac61-metrics-client-ca\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.291260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.290947 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/259aed9c-29fa-4a34-a9e6-8c1466819bb2-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.291260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.291011 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-accelerators-collector-config\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.291595 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.291280 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-wtmp\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.292597 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.292161 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.292597 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.292560 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-textfile\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.292900 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.292874 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/259aed9c-29fa-4a34-a9e6-8c1466819bb2-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.292977 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.292958 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3443e0ca-fb65-47ef-86ba-c3bc535fac61-metrics-client-ca\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.293460 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.293435 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.294484 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.294461 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3443e0ca-fb65-47ef-86ba-c3bc535fac61-node-exporter-tls\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.295248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.295226 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.306030 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.306008 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpwwr\" (UniqueName: \"kubernetes.io/projected/3443e0ca-fb65-47ef-86ba-c3bc535fac61-kube-api-access-qpwwr\") pod \"node-exporter-ztb4x\" (UID: \"3443e0ca-fb65-47ef-86ba-c3bc535fac61\") " pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.309947 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.309903 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nssqz\" (UniqueName: \"kubernetes.io/projected/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-api-access-nssqz\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.419169 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.419060 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ztb4x" Aug 04 16:00:39.796078 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.796038 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:39.799062 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:39.799031 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/259aed9c-29fa-4a34-a9e6-8c1466819bb2-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-kgblb\" (UID: \"259aed9c-29fa-4a34-a9e6-8c1466819bb2\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:40.008163 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:40.008124 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" Aug 04 16:00:40.195652 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:40.195554 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb"] Aug 04 16:00:40.200295 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:00:40.200263 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259aed9c_29fa_4a34_a9e6_8c1466819bb2.slice/crio-3af6077395fa48e1d98121471015915765c4fab9f7016c21828341fe5e6d171c WatchSource:0}: Error finding container 3af6077395fa48e1d98121471015915765c4fab9f7016c21828341fe5e6d171c: Status 404 returned error can't find the container with id 3af6077395fa48e1d98121471015915765c4fab9f7016c21828341fe5e6d171c Aug 04 16:00:40.236544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:40.236480 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ztb4x" event={"ID":"3443e0ca-fb65-47ef-86ba-c3bc535fac61","Type":"ContainerStarted","Data":"6ac3b45f789bea09b254fc612e8eeebfeab786ccdc353fd5f6b9fc294700dae4"} Aug 04 16:00:40.238415 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:40.238234 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" event={"ID":"259aed9c-29fa-4a34-a9e6-8c1466819bb2","Type":"ContainerStarted","Data":"3af6077395fa48e1d98121471015915765c4fab9f7016c21828341fe5e6d171c"} Aug 04 16:00:41.001081 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:41.000983 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 16:00:41.245793 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:41.245753 2556 generic.go:356] "Generic (PLEG): container finished" podID="3443e0ca-fb65-47ef-86ba-c3bc535fac61" containerID="dc1b6136f1c887cc7a8686094e475f3c5a0de359022c6aebea0afd7833428ba1" exitCode=0 Aug 04 16:00:41.246217 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:41.245851 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ztb4x" event={"ID":"3443e0ca-fb65-47ef-86ba-c3bc535fac61","Type":"ContainerDied","Data":"dc1b6136f1c887cc7a8686094e475f3c5a0de359022c6aebea0afd7833428ba1"} Aug 04 16:00:42.119567 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.119523 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6967765754-rbtjx"] Aug 04 16:00:42.123552 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.123521 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.127331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.127102 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Aug 04 16:00:42.127331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.127118 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Aug 04 16:00:42.127331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.127172 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-w7qxq\"" Aug 04 16:00:42.127331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.127257 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Aug 04 16:00:42.128369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.127997 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1pl6p76giu7mm\"" Aug 04 16:00:42.128369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.128065 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Aug 04 16:00:42.128369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.128214 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Aug 04 16:00:42.140119 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.140093 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6967765754-rbtjx"] Aug 04 16:00:42.220608 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.220567 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-tls\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.220817 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.220654 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.220817 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.220711 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/527c1a02-0e18-423b-9d09-f85c4b4dd941-metrics-client-ca\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.220817 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.220773 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkvfz\" (UniqueName: \"kubernetes.io/projected/527c1a02-0e18-423b-9d09-f85c4b4dd941-kube-api-access-kkvfz\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.220817 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.220809 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.221021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.220899 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-grpc-tls\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.221021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.220931 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.221021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.220969 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.252583 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.252543 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ztb4x" event={"ID":"3443e0ca-fb65-47ef-86ba-c3bc535fac61","Type":"ContainerStarted","Data":"39d02ab998e57d74bc0373eaf986384bd2ac602d2875abb12bdca8a4603aeb14"} Aug 04 16:00:42.253062 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.252607 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ztb4x" event={"ID":"3443e0ca-fb65-47ef-86ba-c3bc535fac61","Type":"ContainerStarted","Data":"2fc549aa8bf267cf76a8ce03873cafde270255a459fd14489e829ccf1cd7c728"} Aug 04 16:00:42.276238 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.275896 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ztb4x" podStartSLOduration=2.029871103 podStartE2EDuration="3.275877842s" totalImagesPullingTime="1.169381224s" totalInitContainerRuntime="76.625515ms" isStatefulPod=true podCreationTimestamp="2026-08-04 16:00:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:00:42.275767344 +0000 UTC m=+271.598190825" watchObservedRunningTime="2026-08-04 16:00:42.275877842 +0000 UTC m=+271.598301323" Aug 04 16:00:42.321533 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.321494 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-tls\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.322201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.321548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.322201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.321581 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/527c1a02-0e18-423b-9d09-f85c4b4dd941-metrics-client-ca\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.322201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.321605 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkvfz\" (UniqueName: \"kubernetes.io/projected/527c1a02-0e18-423b-9d09-f85c4b4dd941-kube-api-access-kkvfz\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.322201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.321653 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.322201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.321785 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-grpc-tls\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.322201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.321826 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.322201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.321861 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.325264 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.325215 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/527c1a02-0e18-423b-9d09-f85c4b4dd941-metrics-client-ca\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.327334 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.327302 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-tls\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.327445 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.327423 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.329650 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.329623 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.330686 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.330663 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.331131 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.331094 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-grpc-tls\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.331543 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.331506 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/527c1a02-0e18-423b-9d09-f85c4b4dd941-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.339118 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.339053 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkvfz\" (UniqueName: \"kubernetes.io/projected/527c1a02-0e18-423b-9d09-f85c4b4dd941-kube-api-access-kkvfz\") pod \"thanos-querier-6967765754-rbtjx\" (UID: \"527c1a02-0e18-423b-9d09-f85c4b4dd941\") " pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.436403 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.436324 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:42.615655 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:42.615551 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6967765754-rbtjx"] Aug 04 16:00:42.615885 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:00:42.615855 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527c1a02_0e18_423b_9d09_f85c4b4dd941.slice/crio-e3bc679e7b1f8686aa202a112d25afb34f574757c1a3cd00707522d8250e2b9d WatchSource:0}: Error finding container e3bc679e7b1f8686aa202a112d25afb34f574757c1a3cd00707522d8250e2b9d: Status 404 returned error can't find the container with id e3bc679e7b1f8686aa202a112d25afb34f574757c1a3cd00707522d8250e2b9d Aug 04 16:00:43.258657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.258618 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" event={"ID":"259aed9c-29fa-4a34-a9e6-8c1466819bb2","Type":"ContainerStarted","Data":"d6b8679270cae66b8966b85bd2215dc455fe37febebf7c21751277db31c0a886"} Aug 04 16:00:43.260240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.260212 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" event={"ID":"527c1a02-0e18-423b-9d09-f85c4b4dd941","Type":"ContainerStarted","Data":"e3bc679e7b1f8686aa202a112d25afb34f574757c1a3cd00707522d8250e2b9d"} Aug 04 16:00:43.513797 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.513761 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-69cb84cf77-sq8nt"] Aug 04 16:00:43.517344 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.517318 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.521416 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.521317 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Aug 04 16:00:43.521595 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.521571 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Aug 04 16:00:43.521806 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.521787 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ccsq8istjv7vp\"" Aug 04 16:00:43.522008 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.521988 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Aug 04 16:00:43.522258 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.522157 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-n4ql6\"" Aug 04 16:00:43.523545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.522889 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 04 16:00:43.527989 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.527808 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69cb84cf77-sq8nt"] Aug 04 16:00:43.535678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.535641 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3a5c2c18-b164-4453-8d06-5759538038cd-audit-log\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.535835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.535690 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a5c2c18-b164-4453-8d06-5759538038cd-client-ca-bundle\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.535835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.535769 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3a5c2c18-b164-4453-8d06-5759538038cd-secret-metrics-server-tls\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.535835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.535802 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5c2c18-b164-4453-8d06-5759538038cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.535980 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.535842 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3a5c2c18-b164-4453-8d06-5759538038cd-secret-metrics-server-client-certs\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.535980 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.535877 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6nc2\" (UniqueName: \"kubernetes.io/projected/3a5c2c18-b164-4453-8d06-5759538038cd-kube-api-access-n6nc2\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.535980 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.535924 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3a5c2c18-b164-4453-8d06-5759538038cd-metrics-server-audit-profiles\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.636662 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.636625 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3a5c2c18-b164-4453-8d06-5759538038cd-audit-log\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.636662 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.636662 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a5c2c18-b164-4453-8d06-5759538038cd-client-ca-bundle\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.636927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.636690 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3a5c2c18-b164-4453-8d06-5759538038cd-secret-metrics-server-tls\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.636927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.636719 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5c2c18-b164-4453-8d06-5759538038cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.636927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.636776 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3a5c2c18-b164-4453-8d06-5759538038cd-secret-metrics-server-client-certs\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.636927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.636806 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6nc2\" (UniqueName: \"kubernetes.io/projected/3a5c2c18-b164-4453-8d06-5759538038cd-kube-api-access-n6nc2\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.636927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.636846 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3a5c2c18-b164-4453-8d06-5759538038cd-metrics-server-audit-profiles\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.637524 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.637198 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3a5c2c18-b164-4453-8d06-5759538038cd-audit-log\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.637889 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.637844 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5c2c18-b164-4453-8d06-5759538038cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.638012 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.637983 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3a5c2c18-b164-4453-8d06-5759538038cd-metrics-server-audit-profiles\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.639570 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.639548 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a5c2c18-b164-4453-8d06-5759538038cd-client-ca-bundle\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.639846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.639824 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3a5c2c18-b164-4453-8d06-5759538038cd-secret-metrics-server-client-certs\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.639913 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.639894 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3a5c2c18-b164-4453-8d06-5759538038cd-secret-metrics-server-tls\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.646494 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.646466 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6nc2\" (UniqueName: \"kubernetes.io/projected/3a5c2c18-b164-4453-8d06-5759538038cd-kube-api-access-n6nc2\") pod \"metrics-server-69cb84cf77-sq8nt\" (UID: \"3a5c2c18-b164-4453-8d06-5759538038cd\") " pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.830662 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.830568 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:00:43.994265 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:43.994220 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69cb84cf77-sq8nt"] Aug 04 16:00:43.997708 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:00:43.997674 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5c2c18_b164_4453_8d06_5759538038cd.slice/crio-89c80dc8f2722583c72b4176f3a29466fd514c68a506bbf90ec4a662e6dc5d28 WatchSource:0}: Error finding container 89c80dc8f2722583c72b4176f3a29466fd514c68a506bbf90ec4a662e6dc5d28: Status 404 returned error can't find the container with id 89c80dc8f2722583c72b4176f3a29466fd514c68a506bbf90ec4a662e6dc5d28 Aug 04 16:00:44.268849 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:44.268807 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" event={"ID":"259aed9c-29fa-4a34-a9e6-8c1466819bb2","Type":"ContainerStarted","Data":"266b068e112c2c8eab3e54e5abcf53f97175e53e02b2e49dce2a8e86113ebade"} Aug 04 16:00:44.269329 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:44.268879 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" event={"ID":"259aed9c-29fa-4a34-a9e6-8c1466819bb2","Type":"ContainerStarted","Data":"c9fdf2d6f61434b2552388a3caafb08dc62b4387b4e81dc2ded037e7565f88cc"} Aug 04 16:00:44.270426 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:44.270389 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" event={"ID":"3a5c2c18-b164-4453-8d06-5759538038cd","Type":"ContainerStarted","Data":"89c80dc8f2722583c72b4176f3a29466fd514c68a506bbf90ec4a662e6dc5d28"} Aug 04 16:00:44.296405 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:44.296345 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-kgblb" podStartSLOduration=2.3586772 podStartE2EDuration="5.296321457s" totalImagesPullingTime="2.937644257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:00:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:00:44.294827853 +0000 UTC m=+273.617251332" watchObservedRunningTime="2026-08-04 16:00:44.296321457 +0000 UTC m=+273.618744940" Aug 04 16:00:45.260184 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.258618 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 16:00:45.266075 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.266046 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.271588 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.271051 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Aug 04 16:00:45.271588 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.271055 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Aug 04 16:00:45.275017 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.274143 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Aug 04 16:00:45.276492 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.274359 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Aug 04 16:00:45.276492 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.274460 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-12v101fhj8g8p\"" Aug 04 16:00:45.276492 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.274507 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Aug 04 16:00:45.276492 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.274549 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Aug 04 16:00:45.276492 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.274678 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zrsq8\"" Aug 04 16:00:45.276492 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.274839 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Aug 04 16:00:45.276492 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.275976 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Aug 04 16:00:45.277248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.277057 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Aug 04 16:00:45.277775 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.277424 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Aug 04 16:00:45.277775 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.277490 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Aug 04 16:00:45.278258 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.278037 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Aug 04 16:00:45.278357 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.278331 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Aug 04 16:00:45.283949 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.283029 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 16:00:45.285908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.285873 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Aug 04 16:00:45.353181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353136 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353188 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353217 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353245 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353414 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-config\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353453 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353483 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353537 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353565 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353592 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353622 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzcn9\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-kube-api-access-tzcn9\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353657 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353691 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.354355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353821 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.354355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353937 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.354355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.353976 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.354355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.354000 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.354355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.354038 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.354355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.354094 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455285 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455242 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455285 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455290 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455318 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455335 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455353 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455380 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455433 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455461 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455488 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455866 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455523 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455866 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455560 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455866 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455591 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-config\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.455866 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.455617 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.458395 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.457148 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.458395 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.457711 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.460240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.460210 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.460642 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.460616 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.462749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.456015 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.462749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.461078 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.462749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.461141 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.462749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.461329 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.462749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.461362 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.462749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.461398 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzcn9\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-kube-api-access-tzcn9\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.462749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.461583 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.462749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.461609 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.462749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.462088 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.464090 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.463276 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.471304 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.471255 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.471860 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.471770 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.474339 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.474178 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.474491 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.474454 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.474601 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.474544 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.475007 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.474954 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.475243 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.475221 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.475467 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.475335 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.476306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.476123 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-config\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.476306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.476262 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.476449 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.476306 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.476449 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.476363 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.477589 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.477569 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzcn9\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-kube-api-access-tzcn9\") pod \"prometheus-k8s-0\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:45.580847 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:45.580709 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:00:52.438756 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:52.438709 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 16:00:52.440510 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:00:52.440480 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd08f6ae_8587_47c8_b5e2_269723d77192.slice/crio-e43ef9057fb4ad61902916e154a7b1431493963e8e0d066053ba8af205728c61 WatchSource:0}: Error finding container e43ef9057fb4ad61902916e154a7b1431493963e8e0d066053ba8af205728c61: Status 404 returned error can't find the container with id e43ef9057fb4ad61902916e154a7b1431493963e8e0d066053ba8af205728c61 Aug 04 16:00:53.316553 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:53.316241 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerStarted","Data":"e43ef9057fb4ad61902916e154a7b1431493963e8e0d066053ba8af205728c61"} Aug 04 16:00:53.322394 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:53.321939 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-h82zl" event={"ID":"0e52402b-8a15-412e-87a2-598b4984aae8","Type":"ContainerStarted","Data":"e65e844b32a3817affaf3ee7032d7e1a7fef5f29161dff45bafd665cf5a5641f"} Aug 04 16:00:53.322394 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:53.322337 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-h82zl" Aug 04 16:00:53.333570 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:53.333499 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-h82zl" Aug 04 16:00:53.346071 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:53.345581 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-h82zl" podStartSLOduration=2.203555142 podStartE2EDuration="22.345559564s" totalImagesPullingTime="20.142004422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:00:53.344095417 +0000 UTC m=+282.666518899" watchObservedRunningTime="2026-08-04 16:00:53.345559564 +0000 UTC m=+282.667983046" Aug 04 16:00:54.294012 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:54.293971 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-757cb797ff-mn8b7"] Aug 04 16:00:56.013509 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.013461 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" containerID="cri-o://25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc" gracePeriod=30 Aug 04 16:00:56.341087 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.341043 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" event={"ID":"527c1a02-0e18-423b-9d09-f85c4b4dd941","Type":"ContainerStarted","Data":"f953ddc7c61ae16d4bd00db448e347e28bdcd20f169ac41b50e865290dcb06ce"} Aug 04 16:00:56.341248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.341135 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" event={"ID":"527c1a02-0e18-423b-9d09-f85c4b4dd941","Type":"ContainerStarted","Data":"1f5e081df77c9ab635acfc3e66b27b57748c36397925913336d44485d140daac"} Aug 04 16:00:56.341248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.341175 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" event={"ID":"527c1a02-0e18-423b-9d09-f85c4b4dd941","Type":"ContainerStarted","Data":"e2d4db6e85851947179566ad0a11b48c57bad794257c08b794ca55bf5ac7cd42"} Aug 04 16:00:56.343626 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.343597 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" event={"ID":"3a5c2c18-b164-4453-8d06-5759538038cd","Type":"ContainerStarted","Data":"4a14ee50559af7f3bf95ed80f4a47ca0b8caeb4e465620d51ef30bf5207acd06"} Aug 04 16:00:56.346956 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.346066 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerID="1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e" exitCode=0 Aug 04 16:00:56.346956 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.346146 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerDied","Data":"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e"} Aug 04 16:00:56.352099 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.351981 2556 generic.go:356] "Generic (PLEG): container finished" podID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerID="25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc" exitCode=0 Aug 04 16:00:56.352099 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.352049 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerDied","Data":"25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc"} Aug 04 16:00:56.352099 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.352091 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerStarted","Data":"8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa"} Aug 04 16:00:56.352267 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.352118 2556 scope.go:117] "RemoveContainer" containerID="0f7fec9f248ff7cd67080d7c1b6a3ea8ebcf8e8465452d83a0e65eff49390f69" Aug 04 16:00:56.352267 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.352210 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 16:00:56.371104 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:56.370834 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" podStartSLOduration=2.094492325 podStartE2EDuration="13.370810368s" totalImagesPullingTime="11.276318043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:00:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:00:56.36934983 +0000 UTC m=+285.691773323" watchObservedRunningTime="2026-08-04 16:00:56.370810368 +0000 UTC m=+285.693233849" Aug 04 16:00:57.360824 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:57.360780 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" event={"ID":"527c1a02-0e18-423b-9d09-f85c4b4dd941","Type":"ContainerStarted","Data":"41af4002b9d93ff51c4db4dd5c6c0f9f4aca3fdb442f389ee3cd5f042f45850d"} Aug 04 16:00:58.371719 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:58.371678 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" event={"ID":"527c1a02-0e18-423b-9d09-f85c4b4dd941","Type":"ContainerStarted","Data":"f542907f8d6bdc9e49145db89035cdddb66c77dbf5e653d3d16f6ae41675aee4"} Aug 04 16:00:58.372171 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:58.371764 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" event={"ID":"527c1a02-0e18-423b-9d09-f85c4b4dd941","Type":"ContainerStarted","Data":"caf687090865d597bb485b0ad82b6a316fbe9539d9c396f2e630c27ac5153ac5"} Aug 04 16:00:58.372171 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:58.371864 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:00:58.401515 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:00:58.401441 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" podStartSLOduration=2.331170726 podStartE2EDuration="16.40142307s" totalImagesPullingTime="14.070252344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:00:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:00:58.398868416 +0000 UTC m=+287.721291902" watchObservedRunningTime="2026-08-04 16:00:58.40142307 +0000 UTC m=+287.723846551" Aug 04 16:01:02.399663 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:02.399634 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerStarted","Data":"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b"} Aug 04 16:01:02.400013 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:02.399711 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerStarted","Data":"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613"} Aug 04 16:01:02.400013 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:02.399751 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerStarted","Data":"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42"} Aug 04 16:01:02.400013 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:02.399776 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerStarted","Data":"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee"} Aug 04 16:01:03.407014 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:03.406978 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerStarted","Data":"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0"} Aug 04 16:01:03.407488 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:03.407041 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerStarted","Data":"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac"} Aug 04 16:01:03.457835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:03.457774 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=10.076461061 podStartE2EDuration="18.457748267s" totalImagesPullingTime="8.353760442s" totalInitContainerRuntime="27.526764ms" isStatefulPod=false podCreationTimestamp="2026-08-04 16:00:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:01:03.455786017 +0000 UTC m=+292.778209497" watchObservedRunningTime="2026-08-04 16:01:03.457748267 +0000 UTC m=+292.780171744" Aug 04 16:01:03.830804 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:03.830766 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:01:03.831017 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:03.830838 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:01:04.389093 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:04.389063 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6967765754-rbtjx" Aug 04 16:01:05.581215 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:05.581174 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:01:11.218981 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:11.218953 2556 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 16:01:16.358152 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:16.358124 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 16:01:21.377819 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.377757 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" containerID="cri-o://8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa" gracePeriod=30 Aug 04 16:01:21.627099 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.627074 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 16:01:21.694036 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.693950 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-certificates\") pod \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " Aug 04 16:01:21.694203 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694060 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-installation-pull-secrets\") pod \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " Aug 04 16:01:21.694203 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694110 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-trusted-ca\") pod \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " Aug 04 16:01:21.694203 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694143 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") pod \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " Aug 04 16:01:21.694203 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694165 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f92bfaa-cc35-49fb-af6c-79df9cbec888-ca-trust-extracted\") pod \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " Aug 04 16:01:21.694411 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694210 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njl2n\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-kube-api-access-njl2n\") pod \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " Aug 04 16:01:21.694411 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694241 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-image-registry-private-configuration\") pod \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " Aug 04 16:01:21.694411 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694283 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-bound-sa-token\") pod \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\" (UID: \"0f92bfaa-cc35-49fb-af6c-79df9cbec888\") " Aug 04 16:01:21.694411 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694340 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0f92bfaa-cc35-49fb-af6c-79df9cbec888" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:01:21.694607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694529 2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-certificates\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:01:21.694667 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.694634 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0f92bfaa-cc35-49fb-af6c-79df9cbec888" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:01:21.696760 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.696706 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0f92bfaa-cc35-49fb-af6c-79df9cbec888" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:01:21.696968 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.696798 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0f92bfaa-cc35-49fb-af6c-79df9cbec888" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:01:21.696968 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.696961 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-kube-api-access-njl2n" (OuterVolumeSpecName: "kube-api-access-njl2n") pod "0f92bfaa-cc35-49fb-af6c-79df9cbec888" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888"). InnerVolumeSpecName "kube-api-access-njl2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:01:21.697122 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.697055 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0f92bfaa-cc35-49fb-af6c-79df9cbec888" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:01:21.697269 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.697250 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0f92bfaa-cc35-49fb-af6c-79df9cbec888" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:01:21.703238 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.703208 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f92bfaa-cc35-49fb-af6c-79df9cbec888-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0f92bfaa-cc35-49fb-af6c-79df9cbec888" (UID: "0f92bfaa-cc35-49fb-af6c-79df9cbec888"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:01:21.794908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.794860 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njl2n\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-kube-api-access-njl2n\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:01:21.794908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.794902 2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-image-registry-private-configuration\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:01:21.794908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.794917 2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-bound-sa-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:01:21.795149 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.794929 2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f92bfaa-cc35-49fb-af6c-79df9cbec888-installation-pull-secrets\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:01:21.795149 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.794942 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f92bfaa-cc35-49fb-af6c-79df9cbec888-trusted-ca\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:01:21.795149 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.794954 2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f92bfaa-cc35-49fb-af6c-79df9cbec888-registry-tls\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:01:21.795149 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:21.794964 2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f92bfaa-cc35-49fb-af6c-79df9cbec888-ca-trust-extracted\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:01:22.480957 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.480924 2556 generic.go:356] "Generic (PLEG): container finished" podID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerID="8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa" exitCode=0 Aug 04 16:01:22.481414 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.480966 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerDied","Data":"8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa"} Aug 04 16:01:22.481414 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.480987 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" Aug 04 16:01:22.481414 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.481015 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757cb797ff-mn8b7" event={"ID":"0f92bfaa-cc35-49fb-af6c-79df9cbec888","Type":"ContainerDied","Data":"fc80c200e96a86f7cc324cb123add06a5bd7dd1773df2956ddc6379e04c2a9dd"} Aug 04 16:01:22.481414 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.481042 2556 scope.go:117] "RemoveContainer" containerID="8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa" Aug 04 16:01:22.489489 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.489464 2556 scope.go:117] "RemoveContainer" containerID="25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc" Aug 04 16:01:22.497025 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.497004 2556 scope.go:117] "RemoveContainer" containerID="8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa" Aug 04 16:01:22.497264 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:01:22.497240 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa\": container with ID starting with 8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa not found: ID does not exist" containerID="8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa" Aug 04 16:01:22.497317 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.497273 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa"} err="failed to get container status \"8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa\": rpc error: code = NotFound desc = could not find container \"8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa\": container with ID starting with 8a26c0fd9d7fbb30005b9e172cf4c8548055490570fa3358962a7cdb4b7e69fa not found: ID does not exist" Aug 04 16:01:22.497317 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.497291 2556 scope.go:117] "RemoveContainer" containerID="25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc" Aug 04 16:01:22.497528 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:01:22.497512 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc\": container with ID starting with 25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc not found: ID does not exist" containerID="25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc" Aug 04 16:01:22.497577 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.497533 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc"} err="failed to get container status \"25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc\": rpc error: code = NotFound desc = could not find container \"25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc\": container with ID starting with 25c439e66df9f0a26e63eeb30aede63ad073b7cb82ce36569b20c737071435bc not found: ID does not exist" Aug 04 16:01:22.502460 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.502420 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-757cb797ff-mn8b7"] Aug 04 16:01:22.507077 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:22.507050 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-757cb797ff-mn8b7"] Aug 04 16:01:23.272505 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:23.272472 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" path="/var/lib/kubelet/pods/0f92bfaa-cc35-49fb-af6c-79df9cbec888/volumes" Aug 04 16:01:23.836368 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:23.836338 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:01:23.840162 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:23.840137 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-69cb84cf77-sq8nt" Aug 04 16:01:45.581084 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:45.581048 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:01:45.599597 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:45.599572 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:01:46.592053 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:01:46.592022 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:03.630405 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:03.630305 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 16:02:03.630958 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:03.630665 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="prometheus" containerID="cri-o://7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" gracePeriod=600 Aug 04 16:02:03.631138 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:03.631042 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy-thanos" containerID="cri-o://f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" gracePeriod=600 Aug 04 16:02:03.631716 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:03.631689 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy" containerID="cri-o://6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" gracePeriod=600 Aug 04 16:02:03.631854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:03.631822 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy-web" containerID="cri-o://d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" gracePeriod=600 Aug 04 16:02:03.631907 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:03.631896 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="thanos-sidecar" containerID="cri-o://a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" gracePeriod=600 Aug 04 16:02:03.631987 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:03.631965 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="config-reloader" containerID="cri-o://286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" gracePeriod=600 Aug 04 16:02:03.880821 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:03.880430 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.033487 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033452 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-db\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033487 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033488 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-config-out\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033523 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033548 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-metrics-client-ca\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033566 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-serving-certs-ca-bundle\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033583 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033602 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-1\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033621 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-config\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033642 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-tls-assets\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033670 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-0\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.033711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033699 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-web-config\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033772 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-tls\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033792 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzcn9\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-kube-api-access-tzcn9\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033819 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-thanos-prometheus-http-client-file\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033838 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-trusted-ca-bundle\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033859 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-2\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033876 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-kubelet-serving-ca-bundle\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033902 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-metrics-client-certs\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033927 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-grpc-tls\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033951 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:02:04.034181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.033970 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-kube-rbac-proxy\") pod \"bd08f6ae-8587-47c8-b5e2-269723d77192\" (UID: \"bd08f6ae-8587-47c8-b5e2-269723d77192\") " Aug 04 16:02:04.034648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.034246 2556 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-metrics-client-ca\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.034648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.034545 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:02:04.034648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.034541 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:02:04.034988 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.034959 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:02:04.035079 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.035045 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:02:04.035280 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.035248 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:02:04.035408 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.035261 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:02:04.036456 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.036423 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-config-out" (OuterVolumeSpecName: "config-out") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:02:04.037064 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.037010 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:02:04.037566 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.037535 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-config" (OuterVolumeSpecName: "config") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:02:04.037851 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.037712 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:02:04.037851 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.037784 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:02:04.038008 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.037869 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:02:04.038147 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.038110 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:02:04.038254 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.038191 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:02:04.038926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.038781 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:02:04.039037 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.038881 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-kube-api-access-tzcn9" (OuterVolumeSpecName: "kube-api-access-tzcn9") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "kube-api-access-tzcn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:02:04.039265 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.039244 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:02:04.039495 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.039458 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:02:04.049288 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.049261 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-web-config" (OuterVolumeSpecName: "web-config") pod "bd08f6ae-8587-47c8-b5e2-269723d77192" (UID: "bd08f6ae-8587-47c8-b5e2-269723d77192"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:02:04.135607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135515 2556 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-kube-rbac-proxy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.135607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135559 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-db\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.135607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135570 2556 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd08f6ae-8587-47c8-b5e2-269723d77192-config-out\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.135607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135580 2556 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.135607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135590 2556 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.135607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135599 2556 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.135607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135609 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.135607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135618 2556 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-config\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.135607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135626 2556 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-tls-assets\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135634 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135642 2556 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-web-config\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135650 2556 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135658 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzcn9\" (UniqueName: \"kubernetes.io/projected/bd08f6ae-8587-47c8-b5e2-269723d77192-kube-api-access-tzcn9\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135666 2556 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135674 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135682 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135690 2556 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd08f6ae-8587-47c8-b5e2-269723d77192-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135698 2556 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-metrics-client-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.136005 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.135706 2556 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd08f6ae-8587-47c8-b5e2-269723d77192-secret-grpc-tls\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:02:04.652084 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652053 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerID="f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" exitCode=0 Aug 04 16:02:04.652084 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652081 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerID="6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" exitCode=0 Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652094 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerID="d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" exitCode=0 Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652103 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerID="a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" exitCode=0 Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652111 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerID="286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" exitCode=0 Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652120 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerID="7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" exitCode=0 Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652167 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerDied","Data":"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0"} Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652173 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652242 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerDied","Data":"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac"} Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652280 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerDied","Data":"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b"} Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652308 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerDied","Data":"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613"} Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652346 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerDied","Data":"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42"} Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652392 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerDied","Data":"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee"} Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652427 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd08f6ae-8587-47c8-b5e2-269723d77192","Type":"ContainerDied","Data":"e43ef9057fb4ad61902916e154a7b1431493963e8e0d066053ba8af205728c61"} Aug 04 16:02:04.652545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.652468 2556 scope.go:117] "RemoveContainer" containerID="f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" Aug 04 16:02:04.663267 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.663246 2556 scope.go:117] "RemoveContainer" containerID="6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" Aug 04 16:02:04.670212 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.670192 2556 scope.go:117] "RemoveContainer" containerID="d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" Aug 04 16:02:04.676908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.676888 2556 scope.go:117] "RemoveContainer" containerID="a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" Aug 04 16:02:04.678127 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.678005 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 16:02:04.683091 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.683069 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 16:02:04.684708 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.684689 2556 scope.go:117] "RemoveContainer" containerID="286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" Aug 04 16:02:04.691445 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.691427 2556 scope.go:117] "RemoveContainer" containerID="7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" Aug 04 16:02:04.698993 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.698974 2556 scope.go:117] "RemoveContainer" containerID="1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e" Aug 04 16:02:04.705608 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.705588 2556 scope.go:117] "RemoveContainer" containerID="f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" Aug 04 16:02:04.705850 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:02:04.705832 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": container with ID starting with f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0 not found: ID does not exist" containerID="f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" Aug 04 16:02:04.705912 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.705858 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0"} err="failed to get container status \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": rpc error: code = NotFound desc = could not find container \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": container with ID starting with f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0 not found: ID does not exist" Aug 04 16:02:04.705912 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.705880 2556 scope.go:117] "RemoveContainer" containerID="6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" Aug 04 16:02:04.706108 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:02:04.706095 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": container with ID starting with 6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac not found: ID does not exist" containerID="6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" Aug 04 16:02:04.706148 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.706110 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac"} err="failed to get container status \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": rpc error: code = NotFound desc = could not find container \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": container with ID starting with 6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac not found: ID does not exist" Aug 04 16:02:04.706148 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.706126 2556 scope.go:117] "RemoveContainer" containerID="d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" Aug 04 16:02:04.706350 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:02:04.706333 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": container with ID starting with d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b not found: ID does not exist" containerID="d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" Aug 04 16:02:04.706391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.706357 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b"} err="failed to get container status \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": rpc error: code = NotFound desc = could not find container \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": container with ID starting with d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b not found: ID does not exist" Aug 04 16:02:04.706391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.706374 2556 scope.go:117] "RemoveContainer" containerID="a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" Aug 04 16:02:04.706565 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:02:04.706548 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": container with ID starting with a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613 not found: ID does not exist" containerID="a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" Aug 04 16:02:04.706611 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.706567 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613"} err="failed to get container status \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": rpc error: code = NotFound desc = could not find container \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": container with ID starting with a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613 not found: ID does not exist" Aug 04 16:02:04.706611 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.706578 2556 scope.go:117] "RemoveContainer" containerID="286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" Aug 04 16:02:04.706783 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:02:04.706768 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": container with ID starting with 286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42 not found: ID does not exist" containerID="286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" Aug 04 16:02:04.706848 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.706785 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42"} err="failed to get container status \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": rpc error: code = NotFound desc = could not find container \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": container with ID starting with 286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42 not found: ID does not exist" Aug 04 16:02:04.706848 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.706796 2556 scope.go:117] "RemoveContainer" containerID="7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" Aug 04 16:02:04.706994 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:02:04.706979 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": container with ID starting with 7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee not found: ID does not exist" containerID="7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" Aug 04 16:02:04.707043 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.706996 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee"} err="failed to get container status \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": rpc error: code = NotFound desc = could not find container \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": container with ID starting with 7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee not found: ID does not exist" Aug 04 16:02:04.707043 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.707008 2556 scope.go:117] "RemoveContainer" containerID="1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e" Aug 04 16:02:04.707212 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:02:04.707199 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": container with ID starting with 1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e not found: ID does not exist" containerID="1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e" Aug 04 16:02:04.707249 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.707214 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e"} err="failed to get container status \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": rpc error: code = NotFound desc = could not find container \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": container with ID starting with 1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e not found: ID does not exist" Aug 04 16:02:04.707249 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.707224 2556 scope.go:117] "RemoveContainer" containerID="f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" Aug 04 16:02:04.707403 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.707388 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0"} err="failed to get container status \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": rpc error: code = NotFound desc = could not find container \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": container with ID starting with f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0 not found: ID does not exist" Aug 04 16:02:04.707436 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.707405 2556 scope.go:117] "RemoveContainer" containerID="6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" Aug 04 16:02:04.707639 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.707620 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac"} err="failed to get container status \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": rpc error: code = NotFound desc = could not find container \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": container with ID starting with 6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac not found: ID does not exist" Aug 04 16:02:04.707675 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.707641 2556 scope.go:117] "RemoveContainer" containerID="d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" Aug 04 16:02:04.707870 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.707854 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b"} err="failed to get container status \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": rpc error: code = NotFound desc = could not find container \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": container with ID starting with d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b not found: ID does not exist" Aug 04 16:02:04.707922 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.707871 2556 scope.go:117] "RemoveContainer" containerID="a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" Aug 04 16:02:04.708065 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708049 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613"} err="failed to get container status \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": rpc error: code = NotFound desc = could not find container \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": container with ID starting with a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613 not found: ID does not exist" Aug 04 16:02:04.708113 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708066 2556 scope.go:117] "RemoveContainer" containerID="286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" Aug 04 16:02:04.708246 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708230 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42"} err="failed to get container status \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": rpc error: code = NotFound desc = could not find container \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": container with ID starting with 286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42 not found: ID does not exist" Aug 04 16:02:04.708293 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708246 2556 scope.go:117] "RemoveContainer" containerID="7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" Aug 04 16:02:04.708466 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708447 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee"} err="failed to get container status \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": rpc error: code = NotFound desc = could not find container \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": container with ID starting with 7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee not found: ID does not exist" Aug 04 16:02:04.708518 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708465 2556 scope.go:117] "RemoveContainer" containerID="1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e" Aug 04 16:02:04.708643 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708625 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e"} err="failed to get container status \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": rpc error: code = NotFound desc = could not find container \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": container with ID starting with 1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e not found: ID does not exist" Aug 04 16:02:04.708680 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708644 2556 scope.go:117] "RemoveContainer" containerID="f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" Aug 04 16:02:04.708826 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708810 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0"} err="failed to get container status \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": rpc error: code = NotFound desc = could not find container \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": container with ID starting with f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0 not found: ID does not exist" Aug 04 16:02:04.708872 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.708827 2556 scope.go:117] "RemoveContainer" containerID="6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" Aug 04 16:02:04.709021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709007 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac"} err="failed to get container status \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": rpc error: code = NotFound desc = could not find container \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": container with ID starting with 6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac not found: ID does not exist" Aug 04 16:02:04.709062 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709020 2556 scope.go:117] "RemoveContainer" containerID="d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" Aug 04 16:02:04.709202 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709186 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b"} err="failed to get container status \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": rpc error: code = NotFound desc = could not find container \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": container with ID starting with d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b not found: ID does not exist" Aug 04 16:02:04.709238 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709202 2556 scope.go:117] "RemoveContainer" containerID="a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" Aug 04 16:02:04.709376 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709360 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613"} err="failed to get container status \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": rpc error: code = NotFound desc = could not find container \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": container with ID starting with a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613 not found: ID does not exist" Aug 04 16:02:04.709418 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709375 2556 scope.go:117] "RemoveContainer" containerID="286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" Aug 04 16:02:04.709544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709528 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42"} err="failed to get container status \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": rpc error: code = NotFound desc = could not find container \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": container with ID starting with 286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42 not found: ID does not exist" Aug 04 16:02:04.709544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709543 2556 scope.go:117] "RemoveContainer" containerID="7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" Aug 04 16:02:04.709712 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709697 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee"} err="failed to get container status \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": rpc error: code = NotFound desc = could not find container \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": container with ID starting with 7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee not found: ID does not exist" Aug 04 16:02:04.709776 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709711 2556 scope.go:117] "RemoveContainer" containerID="1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e" Aug 04 16:02:04.709892 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709873 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e"} err="failed to get container status \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": rpc error: code = NotFound desc = could not find container \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": container with ID starting with 1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e not found: ID does not exist" Aug 04 16:02:04.709933 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.709892 2556 scope.go:117] "RemoveContainer" containerID="f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" Aug 04 16:02:04.710049 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710034 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0"} err="failed to get container status \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": rpc error: code = NotFound desc = could not find container \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": container with ID starting with f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0 not found: ID does not exist" Aug 04 16:02:04.710091 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710050 2556 scope.go:117] "RemoveContainer" containerID="6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" Aug 04 16:02:04.710248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710233 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac"} err="failed to get container status \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": rpc error: code = NotFound desc = could not find container \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": container with ID starting with 6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac not found: ID does not exist" Aug 04 16:02:04.710248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710247 2556 scope.go:117] "RemoveContainer" containerID="d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" Aug 04 16:02:04.710428 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710414 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b"} err="failed to get container status \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": rpc error: code = NotFound desc = could not find container \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": container with ID starting with d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b not found: ID does not exist" Aug 04 16:02:04.710468 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710428 2556 scope.go:117] "RemoveContainer" containerID="a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" Aug 04 16:02:04.710589 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710574 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613"} err="failed to get container status \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": rpc error: code = NotFound desc = could not find container \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": container with ID starting with a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613 not found: ID does not exist" Aug 04 16:02:04.710627 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710589 2556 scope.go:117] "RemoveContainer" containerID="286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" Aug 04 16:02:04.710797 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710782 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42"} err="failed to get container status \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": rpc error: code = NotFound desc = could not find container \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": container with ID starting with 286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42 not found: ID does not exist" Aug 04 16:02:04.710844 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710797 2556 scope.go:117] "RemoveContainer" containerID="7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" Aug 04 16:02:04.710953 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710939 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee"} err="failed to get container status \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": rpc error: code = NotFound desc = could not find container \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": container with ID starting with 7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee not found: ID does not exist" Aug 04 16:02:04.710998 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.710954 2556 scope.go:117] "RemoveContainer" containerID="1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e" Aug 04 16:02:04.711107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711091 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e"} err="failed to get container status \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": rpc error: code = NotFound desc = could not find container \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": container with ID starting with 1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e not found: ID does not exist" Aug 04 16:02:04.711172 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711108 2556 scope.go:117] "RemoveContainer" containerID="f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" Aug 04 16:02:04.711305 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711290 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0"} err="failed to get container status \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": rpc error: code = NotFound desc = could not find container \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": container with ID starting with f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0 not found: ID does not exist" Aug 04 16:02:04.711342 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711305 2556 scope.go:117] "RemoveContainer" containerID="6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" Aug 04 16:02:04.711495 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711481 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac"} err="failed to get container status \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": rpc error: code = NotFound desc = could not find container \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": container with ID starting with 6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac not found: ID does not exist" Aug 04 16:02:04.711531 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711496 2556 scope.go:117] "RemoveContainer" containerID="d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" Aug 04 16:02:04.711733 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711701 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b"} err="failed to get container status \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": rpc error: code = NotFound desc = could not find container \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": container with ID starting with d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b not found: ID does not exist" Aug 04 16:02:04.711784 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711732 2556 scope.go:117] "RemoveContainer" containerID="a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" Aug 04 16:02:04.711947 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711931 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613"} err="failed to get container status \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": rpc error: code = NotFound desc = could not find container \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": container with ID starting with a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613 not found: ID does not exist" Aug 04 16:02:04.711995 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.711947 2556 scope.go:117] "RemoveContainer" containerID="286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" Aug 04 16:02:04.712120 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712100 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42"} err="failed to get container status \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": rpc error: code = NotFound desc = could not find container \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": container with ID starting with 286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42 not found: ID does not exist" Aug 04 16:02:04.712187 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712123 2556 scope.go:117] "RemoveContainer" containerID="7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" Aug 04 16:02:04.712316 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712301 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee"} err="failed to get container status \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": rpc error: code = NotFound desc = could not find container \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": container with ID starting with 7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee not found: ID does not exist" Aug 04 16:02:04.712364 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712315 2556 scope.go:117] "RemoveContainer" containerID="1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e" Aug 04 16:02:04.712475 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712459 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e"} err="failed to get container status \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": rpc error: code = NotFound desc = could not find container \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": container with ID starting with 1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e not found: ID does not exist" Aug 04 16:02:04.712517 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712475 2556 scope.go:117] "RemoveContainer" containerID="f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0" Aug 04 16:02:04.712631 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712615 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0"} err="failed to get container status \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": rpc error: code = NotFound desc = could not find container \"f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0\": container with ID starting with f7bc0342c4fcea7325cb2dfe07429b36e14e71865bbcc559cc3399823ecd70d0 not found: ID does not exist" Aug 04 16:02:04.712671 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712632 2556 scope.go:117] "RemoveContainer" containerID="6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac" Aug 04 16:02:04.712845 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712829 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac"} err="failed to get container status \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": rpc error: code = NotFound desc = could not find container \"6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac\": container with ID starting with 6445db70fd178d45fb32d486f25cbc6bd4dc7892e7f232358de32c6da8e4caac not found: ID does not exist" Aug 04 16:02:04.712899 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.712844 2556 scope.go:117] "RemoveContainer" containerID="d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b" Aug 04 16:02:04.713024 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.713008 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b"} err="failed to get container status \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": rpc error: code = NotFound desc = could not find container \"d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b\": container with ID starting with d4e828bf204a6e584ff5e3dc554696711ea26a2a1868cdc369eb461f4492e94b not found: ID does not exist" Aug 04 16:02:04.713024 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.713024 2556 scope.go:117] "RemoveContainer" containerID="a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613" Aug 04 16:02:04.713215 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.713196 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613"} err="failed to get container status \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": rpc error: code = NotFound desc = could not find container \"a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613\": container with ID starting with a0ca3025f1f3876127e4f85f956be834a69e80d41c36eb19a753a31cef013613 not found: ID does not exist" Aug 04 16:02:04.713280 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.713217 2556 scope.go:117] "RemoveContainer" containerID="286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42" Aug 04 16:02:04.713412 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.713394 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42"} err="failed to get container status \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": rpc error: code = NotFound desc = could not find container \"286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42\": container with ID starting with 286b49b13e57224c9898b398628cd6ed7a263c836bffd461d3c178a4dde43b42 not found: ID does not exist" Aug 04 16:02:04.713456 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.713411 2556 scope.go:117] "RemoveContainer" containerID="7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee" Aug 04 16:02:04.713607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.713589 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee"} err="failed to get container status \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": rpc error: code = NotFound desc = could not find container \"7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee\": container with ID starting with 7944495cc65dc3ea2d5ad6d27dfc06fe32f8f3a0fa3898904417b275a13cacee not found: ID does not exist" Aug 04 16:02:04.713675 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.713608 2556 scope.go:117] "RemoveContainer" containerID="1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e" Aug 04 16:02:04.713910 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.713895 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e"} err="failed to get container status \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": rpc error: code = NotFound desc = could not find container \"1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e\": container with ID starting with 1bc09cedd48cf6efb03993dbf2ec6f5d5f5b2890cc3596991b1e7c093d174a7e not found: ID does not exist" Aug 04 16:02:04.720532 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720510 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 16:02:04.720805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720792 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="prometheus" Aug 04 16:02:04.720854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720807 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="prometheus" Aug 04 16:02:04.720854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720816 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.720854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720822 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.720854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720828 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="config-reloader" Aug 04 16:02:04.720854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720834 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="config-reloader" Aug 04 16:02:04.720854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720841 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy-web" Aug 04 16:02:04.720854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720846 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy-web" Aug 04 16:02:04.720854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720853 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy-thanos" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720858 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy-thanos" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720868 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720873 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720882 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720904 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720911 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="init-config-reloader" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720916 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="init-config-reloader" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720923 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720929 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720957 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="thanos-sidecar" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720962 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="thanos-sidecar" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720968 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.720973 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721022 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721029 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721036 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy-web" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721042 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="kube-rbac-proxy-thanos" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721047 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721052 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721056 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="config-reloader" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721061 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="prometheus" Aug 04 16:02:04.721069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721067 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" containerName="thanos-sidecar" Aug 04 16:02:04.721641 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.721164 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f92bfaa-cc35-49fb-af6c-79df9cbec888" containerName="registry" Aug 04 16:02:04.725870 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.725855 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.728670 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.728654 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Aug 04 16:02:04.728867 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.728847 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Aug 04 16:02:04.728957 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.728942 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Aug 04 16:02:04.729008 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.728954 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Aug 04 16:02:04.729008 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.728983 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Aug 04 16:02:04.729150 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.729127 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Aug 04 16:02:04.729327 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.729311 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zrsq8\"" Aug 04 16:02:04.729401 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.729342 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Aug 04 16:02:04.729401 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.729373 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Aug 04 16:02:04.729550 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.729533 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Aug 04 16:02:04.729649 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.729634 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Aug 04 16:02:04.729872 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.729856 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Aug 04 16:02:04.729948 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.729858 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Aug 04 16:02:04.730786 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.730772 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-12v101fhj8g8p\"" Aug 04 16:02:04.732631 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.732604 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Aug 04 16:02:04.734705 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.734684 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Aug 04 16:02:04.744590 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.744568 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 16:02:04.841256 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841222 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841256 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841256 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841473 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841280 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841473 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841303 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841473 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841320 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841473 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841344 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841473 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841361 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdzhq\" (UniqueName: \"kubernetes.io/projected/6c5eb557-35d6-4de5-9111-ca54d47cd01f-kube-api-access-jdzhq\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841473 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841378 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841473 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841427 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841486 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841507 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-web-config\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841525 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c5eb557-35d6-4de5-9111-ca54d47cd01f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841539 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841559 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841574 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841588 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841627 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841669 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841711 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-config\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.841805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.841779 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c5eb557-35d6-4de5-9111-ca54d47cd01f-config-out\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.942607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942512 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.942607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942555 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.942607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942589 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.942908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942617 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-web-config\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.942908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942638 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c5eb557-35d6-4de5-9111-ca54d47cd01f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.942908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942658 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.942908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942689 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.942908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942713 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.942908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942891 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942931 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942965 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.942999 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-config\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943035 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c5eb557-35d6-4de5-9111-ca54d47cd01f-config-out\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943071 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943096 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943137 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943164 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943193 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943642 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943241 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943642 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943291 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdzhq\" (UniqueName: \"kubernetes.io/projected/6c5eb557-35d6-4de5-9111-ca54d47cd01f-kube-api-access-jdzhq\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943917 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943094 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.943917 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.943789 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.944777 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.944748 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.945450 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.945385 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.946166 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.946138 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.947462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.946391 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.947462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.946663 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.947462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.946878 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.947462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.946939 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.947462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.947073 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-config\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.947462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.947270 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.947462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.947433 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.947848 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.947817 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6c5eb557-35d6-4de5-9111-ca54d47cd01f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.947848 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.947831 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.948366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.948099 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c5eb557-35d6-4de5-9111-ca54d47cd01f-config-out\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.948366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.948281 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.948822 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.948802 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.949071 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.949048 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c5eb557-35d6-4de5-9111-ca54d47cd01f-web-config\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.949347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.949330 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c5eb557-35d6-4de5-9111-ca54d47cd01f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:04.952597 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:04.952574 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdzhq\" (UniqueName: \"kubernetes.io/projected/6c5eb557-35d6-4de5-9111-ca54d47cd01f-kube-api-access-jdzhq\") pod \"prometheus-k8s-0\" (UID: \"6c5eb557-35d6-4de5-9111-ca54d47cd01f\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:05.035138 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:05.035096 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:02:05.182032 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:05.182006 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 16:02:05.184748 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:02:05.184707 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5eb557_35d6_4de5_9111_ca54d47cd01f.slice/crio-f78e194ee694cb2e5415bedf20670c16ca04b92576a7ba1cc00a595c2bb49fa7 WatchSource:0}: Error finding container f78e194ee694cb2e5415bedf20670c16ca04b92576a7ba1cc00a595c2bb49fa7: Status 404 returned error can't find the container with id f78e194ee694cb2e5415bedf20670c16ca04b92576a7ba1cc00a595c2bb49fa7 Aug 04 16:02:05.272376 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:05.272345 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd08f6ae-8587-47c8-b5e2-269723d77192" path="/var/lib/kubelet/pods/bd08f6ae-8587-47c8-b5e2-269723d77192/volumes" Aug 04 16:02:05.656872 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:05.656842 2556 generic.go:356] "Generic (PLEG): container finished" podID="6c5eb557-35d6-4de5-9111-ca54d47cd01f" containerID="b068b965a439ba09ffd32315aa0d3f54d05a5786f0b1d08fc3025aa3fb93725d" exitCode=0 Aug 04 16:02:05.657293 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:05.656942 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c5eb557-35d6-4de5-9111-ca54d47cd01f","Type":"ContainerDied","Data":"b068b965a439ba09ffd32315aa0d3f54d05a5786f0b1d08fc3025aa3fb93725d"} Aug 04 16:02:05.657293 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:05.657018 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c5eb557-35d6-4de5-9111-ca54d47cd01f","Type":"ContainerStarted","Data":"f78e194ee694cb2e5415bedf20670c16ca04b92576a7ba1cc00a595c2bb49fa7"} Aug 04 16:02:06.664413 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:06.664380 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c5eb557-35d6-4de5-9111-ca54d47cd01f","Type":"ContainerStarted","Data":"812a42b7a2a526462e337ee27b620e441557023fa0b27d5fd52ce38f31299704"} Aug 04 16:02:06.664803 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:06.664443 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c5eb557-35d6-4de5-9111-ca54d47cd01f","Type":"ContainerStarted","Data":"0e02b2d5bdd19b693a3c701e8a69fead29965fa1c54d98823da93f7425a601d9"} Aug 04 16:02:06.664803 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:06.664470 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c5eb557-35d6-4de5-9111-ca54d47cd01f","Type":"ContainerStarted","Data":"d6b149218f758c011260e5ff13b7bf00b7b62f8dacafde29da2f56a97a323805"} Aug 04 16:02:06.664803 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:06.664494 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c5eb557-35d6-4de5-9111-ca54d47cd01f","Type":"ContainerStarted","Data":"1e5d79d25618f4fe61219cc6626f907d70d19e0f4820f3fca78b5d19e419dedc"} Aug 04 16:02:06.664803 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:06.664519 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c5eb557-35d6-4de5-9111-ca54d47cd01f","Type":"ContainerStarted","Data":"027113064a518392828bb7cad3b1e4a6e06c36f7c70548313cfb6eb8f692539a"} Aug 04 16:02:06.664803 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:06.664545 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c5eb557-35d6-4de5-9111-ca54d47cd01f","Type":"ContainerStarted","Data":"a11320710f560b0bffcde022bbba62e36e7f214bbbefd1ff9432bd7267c6f592"} Aug 04 16:02:06.705310 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:06.705251 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.686401817 podStartE2EDuration="2.705232961s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.831144ms" isStatefulPod=false podCreationTimestamp="2026-08-04 16:02:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:02:06.703061734 +0000 UTC m=+356.025485229" watchObservedRunningTime="2026-08-04 16:02:06.705232961 +0000 UTC m=+356.027656441" Aug 04 16:02:10.035331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:02:10.035290 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:03:05.035782 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:03:05.035716 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:03:05.051439 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:03:05.051414 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:03:05.909763 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:03:05.909734 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 16:05:26.933201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:26.933117 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-pqvkv"] Aug 04 16:05:26.936469 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:26.936450 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:26.944306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:26.944279 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Aug 04 16:05:26.944476 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:26.944456 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Aug 04 16:05:26.944523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:26.944473 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-z6m2t\"" Aug 04 16:05:26.964369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:26.964334 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-pqvkv"] Aug 04 16:05:27.028037 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.028000 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d19e891c-edbc-47bb-8f63-0fba3bfe8ada-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-pqvkv\" (UID: \"d19e891c-edbc-47bb-8f63-0fba3bfe8ada\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:27.028037 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.028039 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cgr4\" (UniqueName: \"kubernetes.io/projected/d19e891c-edbc-47bb-8f63-0fba3bfe8ada-kube-api-access-4cgr4\") pod \"cert-manager-webhook-6486ffd997-pqvkv\" (UID: \"d19e891c-edbc-47bb-8f63-0fba3bfe8ada\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:27.129123 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.129068 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d19e891c-edbc-47bb-8f63-0fba3bfe8ada-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-pqvkv\" (UID: \"d19e891c-edbc-47bb-8f63-0fba3bfe8ada\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:27.129123 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.129121 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cgr4\" (UniqueName: \"kubernetes.io/projected/d19e891c-edbc-47bb-8f63-0fba3bfe8ada-kube-api-access-4cgr4\") pod \"cert-manager-webhook-6486ffd997-pqvkv\" (UID: \"d19e891c-edbc-47bb-8f63-0fba3bfe8ada\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:27.140418 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.140381 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d19e891c-edbc-47bb-8f63-0fba3bfe8ada-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-pqvkv\" (UID: \"d19e891c-edbc-47bb-8f63-0fba3bfe8ada\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:27.140547 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.140445 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cgr4\" (UniqueName: \"kubernetes.io/projected/d19e891c-edbc-47bb-8f63-0fba3bfe8ada-kube-api-access-4cgr4\") pod \"cert-manager-webhook-6486ffd997-pqvkv\" (UID: \"d19e891c-edbc-47bb-8f63-0fba3bfe8ada\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:27.245468 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.245433 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:27.398367 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.398335 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-pqvkv"] Aug 04 16:05:27.401470 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:05:27.401439 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd19e891c_edbc_47bb_8f63_0fba3bfe8ada.slice/crio-8acae26d2ea6a0d80d5ca482b7d8c15893a8a744da093e45b4a609be29629613 WatchSource:0}: Error finding container 8acae26d2ea6a0d80d5ca482b7d8c15893a8a744da093e45b4a609be29629613: Status 404 returned error can't find the container with id 8acae26d2ea6a0d80d5ca482b7d8c15893a8a744da093e45b4a609be29629613 Aug 04 16:05:27.403317 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.403302 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 16:05:27.436663 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:27.436621 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" event={"ID":"d19e891c-edbc-47bb-8f63-0fba3bfe8ada","Type":"ContainerStarted","Data":"8acae26d2ea6a0d80d5ca482b7d8c15893a8a744da093e45b4a609be29629613"} Aug 04 16:05:31.456501 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:31.456460 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" event={"ID":"d19e891c-edbc-47bb-8f63-0fba3bfe8ada","Type":"ContainerStarted","Data":"e06f3efacd7116f262ad74ca81427dee34172919ec6dfa4557260b3e88d262e5"} Aug 04 16:05:31.457017 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:31.456594 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:31.477583 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:31.477532 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" podStartSLOduration=2.278391856 podStartE2EDuration="5.477514938s" totalImagesPullingTime="3.199123082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:05:31.475873071 +0000 UTC m=+560.798296581" watchObservedRunningTime="2026-08-04 16:05:31.477514938 +0000 UTC m=+560.799938418" Aug 04 16:05:37.463283 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:37.463254 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-pqvkv" Aug 04 16:05:58.470859 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.470820 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz"] Aug 04 16:05:58.477956 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.477912 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.482096 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.482072 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 16:05:58.482096 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.482091 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 16:05:58.483680 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.483661 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 16:05:58.486965 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.486942 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ft4sg\"" Aug 04 16:05:58.487127 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.487109 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 16:05:58.500398 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.500367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/114027f0-89cf-4672-9f5e-44d5dab1e346-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.500530 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.500416 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/114027f0-89cf-4672-9f5e-44d5dab1e346-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.500530 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.500441 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/114027f0-89cf-4672-9f5e-44d5dab1e346-manifests\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.500530 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.500465 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkqhv\" (UniqueName: \"kubernetes.io/projected/114027f0-89cf-4672-9f5e-44d5dab1e346-kube-api-access-pkqhv\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.500530 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.500498 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/114027f0-89cf-4672-9f5e-44d5dab1e346-tmp\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.590969 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.590920 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz"] Aug 04 16:05:58.601261 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.601222 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/114027f0-89cf-4672-9f5e-44d5dab1e346-tmp\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.601447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.601288 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/114027f0-89cf-4672-9f5e-44d5dab1e346-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.601447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.601346 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/114027f0-89cf-4672-9f5e-44d5dab1e346-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.601447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.601367 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/114027f0-89cf-4672-9f5e-44d5dab1e346-manifests\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.601447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.601393 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkqhv\" (UniqueName: \"kubernetes.io/projected/114027f0-89cf-4672-9f5e-44d5dab1e346-kube-api-access-pkqhv\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.602031 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.601992 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/114027f0-89cf-4672-9f5e-44d5dab1e346-manifests\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.603702 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.603680 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/114027f0-89cf-4672-9f5e-44d5dab1e346-tmp\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.604176 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.604156 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/114027f0-89cf-4672-9f5e-44d5dab1e346-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.604397 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.604376 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/114027f0-89cf-4672-9f5e-44d5dab1e346-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.615352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.615322 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkqhv\" (UniqueName: \"kubernetes.io/projected/114027f0-89cf-4672-9f5e-44d5dab1e346-kube-api-access-pkqhv\") pod \"opendatahub-operator-controller-manager-7dfb598967-fv7cz\" (UID: \"114027f0-89cf-4672-9f5e-44d5dab1e346\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.789022 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.788977 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:05:58.930444 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:58.930409 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz"] Aug 04 16:05:58.933973 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:05:58.933937 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114027f0_89cf_4672_9f5e_44d5dab1e346.slice/crio-071b76b6587d5e1d86542f52047e1b4766a55a53447bea263fb049c8e9e2ebf2 WatchSource:0}: Error finding container 071b76b6587d5e1d86542f52047e1b4766a55a53447bea263fb049c8e9e2ebf2: Status 404 returned error can't find the container with id 071b76b6587d5e1d86542f52047e1b4766a55a53447bea263fb049c8e9e2ebf2 Aug 04 16:05:59.573413 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:05:59.573377 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" event={"ID":"114027f0-89cf-4672-9f5e-44d5dab1e346","Type":"ContainerStarted","Data":"071b76b6587d5e1d86542f52047e1b4766a55a53447bea263fb049c8e9e2ebf2"} Aug 04 16:06:02.589481 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:02.589446 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" event={"ID":"114027f0-89cf-4672-9f5e-44d5dab1e346","Type":"ContainerStarted","Data":"8d1bbed6488f9020ed67582edcef320fafbe3c4aa45115acad366740bc808ca4"} Aug 04 16:06:03.595392 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:03.595361 2556 generic.go:356] "Generic (PLEG): container finished" podID="114027f0-89cf-4672-9f5e-44d5dab1e346" containerID="8d1bbed6488f9020ed67582edcef320fafbe3c4aa45115acad366740bc808ca4" exitCode=0 Aug 04 16:06:03.595742 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:03.595419 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" event={"ID":"114027f0-89cf-4672-9f5e-44d5dab1e346","Type":"ContainerDied","Data":"8d1bbed6488f9020ed67582edcef320fafbe3c4aa45115acad366740bc808ca4"} Aug 04 16:06:06.610547 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:06.610509 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" event={"ID":"114027f0-89cf-4672-9f5e-44d5dab1e346","Type":"ContainerStarted","Data":"3020f2ce2d9f547fd4bdbb9eace56a13fb68a3adf5d50c921ff6dfe14e324158"} Aug 04 16:06:06.611043 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:06.610622 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:06:06.635558 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:06.635463 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" podStartSLOduration=1.415820674 podStartE2EDuration="8.635444393s" totalImagesPullingTime="6.178445311s" totalInitContainerRuntime="1.041178408s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:05:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:06:06.634039417 +0000 UTC m=+595.956462924" watchObservedRunningTime="2026-08-04 16:06:06.635444393 +0000 UTC m=+595.957867874" Aug 04 16:06:17.617776 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:17.617745 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7dfb598967-fv7cz" Aug 04 16:06:29.357621 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.357586 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k"] Aug 04 16:06:29.361476 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.361457 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.366228 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.366172 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 16:06:29.367280 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.367249 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 16:06:29.367560 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.367542 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 16:06:29.367803 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.367787 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-cpk5p\"" Aug 04 16:06:29.387524 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.387489 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k"] Aug 04 16:06:29.449306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.449271 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.449306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.449312 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.449533 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.449339 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.449533 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.449411 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hcgc\" (UniqueName: \"kubernetes.io/projected/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-kube-api-access-8hcgc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.449533 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.449445 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.449533 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.449470 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.449715 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.449534 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.449715 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.449560 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.449715 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.449579 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550151 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550110 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550346 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550158 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hcgc\" (UniqueName: \"kubernetes.io/projected/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-kube-api-access-8hcgc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550346 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550190 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550346 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550217 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550346 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550271 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550346 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550307 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550346 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550336 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550655 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550370 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550655 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550408 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550776 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550715 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550838 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550772 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550895 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550851 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.550938 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.550921 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.551123 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.551097 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.552737 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.552697 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.552924 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.552908 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.558974 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.558951 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc"] Aug 04 16:06:29.563095 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.563077 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.575747 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.575709 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 04 16:06:29.576124 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.576109 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-l5rrd\"" Aug 04 16:06:29.576856 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.576841 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 04 16:06:29.576913 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.576896 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 04 16:06:29.576948 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.576935 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 04 16:06:29.583594 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.583574 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 04 16:06:29.585581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.585563 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.585874 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.585854 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hcgc\" (UniqueName: \"kubernetes.io/projected/8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227-kube-api-access-8hcgc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k\" (UID: \"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.608241 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.608167 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc"] Aug 04 16:06:29.651112 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.651079 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a20b218-12f0-4e7b-ba34-579476907d4f-cert\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.651285 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.651120 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a20b218-12f0-4e7b-ba34-579476907d4f-manager-config\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.651285 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.651193 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqx49\" (UniqueName: \"kubernetes.io/projected/7a20b218-12f0-4e7b-ba34-579476907d4f-kube-api-access-qqx49\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.651285 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.651263 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a20b218-12f0-4e7b-ba34-579476907d4f-metrics-cert\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.676357 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.676316 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:29.754323 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.754013 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a20b218-12f0-4e7b-ba34-579476907d4f-cert\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.754323 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.754066 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a20b218-12f0-4e7b-ba34-579476907d4f-manager-config\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.754323 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.754105 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqx49\" (UniqueName: \"kubernetes.io/projected/7a20b218-12f0-4e7b-ba34-579476907d4f-kube-api-access-qqx49\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.754323 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.754139 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a20b218-12f0-4e7b-ba34-579476907d4f-metrics-cert\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.755171 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.755145 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a20b218-12f0-4e7b-ba34-579476907d4f-manager-config\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.757513 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.757145 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a20b218-12f0-4e7b-ba34-579476907d4f-cert\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.757513 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.757482 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a20b218-12f0-4e7b-ba34-579476907d4f-metrics-cert\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.769923 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.769854 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqx49\" (UniqueName: \"kubernetes.io/projected/7a20b218-12f0-4e7b-ba34-579476907d4f-kube-api-access-qqx49\") pod \"lws-controller-manager-5967759b96-mbcvc\" (UID: \"7a20b218-12f0-4e7b-ba34-579476907d4f\") " pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:29.867514 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:06:29.867448 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b5cd07d_d8e3_45fa_9d54_b6e0fb5bc227.slice/crio-10ed03bf68ba9166ae4d2bfe6608719b9e528d940a258909511b4c90e1bf76c7 WatchSource:0}: Error finding container 10ed03bf68ba9166ae4d2bfe6608719b9e528d940a258909511b4c90e1bf76c7: Status 404 returned error can't find the container with id 10ed03bf68ba9166ae4d2bfe6608719b9e528d940a258909511b4c90e1bf76c7 Aug 04 16:06:29.869122 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.869026 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k"] Aug 04 16:06:29.875979 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:29.875955 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:30.019816 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:30.019783 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc"] Aug 04 16:06:30.025752 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:06:30.025702 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a20b218_12f0_4e7b_ba34_579476907d4f.slice/crio-9b563bd08c49e147dcfd7a5aef08899078740af5bae02d121ecda9c2df29290f WatchSource:0}: Error finding container 9b563bd08c49e147dcfd7a5aef08899078740af5bae02d121ecda9c2df29290f: Status 404 returned error can't find the container with id 9b563bd08c49e147dcfd7a5aef08899078740af5bae02d121ecda9c2df29290f Aug 04 16:06:30.716273 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:30.716234 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" event={"ID":"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227","Type":"ContainerStarted","Data":"10ed03bf68ba9166ae4d2bfe6608719b9e528d940a258909511b4c90e1bf76c7"} Aug 04 16:06:30.717365 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:30.717338 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" event={"ID":"7a20b218-12f0-4e7b-ba34-579476907d4f","Type":"ContainerStarted","Data":"9b563bd08c49e147dcfd7a5aef08899078740af5bae02d121ecda9c2df29290f"} Aug 04 16:06:32.496233 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.496104 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5z45r"] Aug 04 16:06:32.499950 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.499928 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:32.504137 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.504104 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Aug 04 16:06:32.504293 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.504263 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-d2lks\"" Aug 04 16:06:32.515531 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.515507 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5z45r"] Aug 04 16:06:32.583187 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.583127 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18303a78-30fe-4a3d-bbc6-d30bc4604c51-cert\") pod \"kserve-controller-manager-d6f4b65b8-5z45r\" (UID: \"18303a78-30fe-4a3d-bbc6-d30bc4604c51\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:32.583187 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.583183 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x5l2\" (UniqueName: \"kubernetes.io/projected/18303a78-30fe-4a3d-bbc6-d30bc4604c51-kube-api-access-4x5l2\") pod \"kserve-controller-manager-d6f4b65b8-5z45r\" (UID: \"18303a78-30fe-4a3d-bbc6-d30bc4604c51\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:32.684745 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.684619 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18303a78-30fe-4a3d-bbc6-d30bc4604c51-cert\") pod \"kserve-controller-manager-d6f4b65b8-5z45r\" (UID: \"18303a78-30fe-4a3d-bbc6-d30bc4604c51\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:32.684745 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.684665 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x5l2\" (UniqueName: \"kubernetes.io/projected/18303a78-30fe-4a3d-bbc6-d30bc4604c51-kube-api-access-4x5l2\") pod \"kserve-controller-manager-d6f4b65b8-5z45r\" (UID: \"18303a78-30fe-4a3d-bbc6-d30bc4604c51\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:32.684994 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:06:32.684915 2556 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Aug 04 16:06:32.684994 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:06:32.684988 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18303a78-30fe-4a3d-bbc6-d30bc4604c51-cert podName:18303a78-30fe-4a3d-bbc6-d30bc4604c51 nodeName:}" failed. No retries permitted until 2026-08-04 16:06:33.18496925 +0000 UTC m=+622.507392713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/18303a78-30fe-4a3d-bbc6-d30bc4604c51-cert") pod "kserve-controller-manager-d6f4b65b8-5z45r" (UID: "18303a78-30fe-4a3d-bbc6-d30bc4604c51") : secret "kserve-webhook-server-cert" not found Aug 04 16:06:32.702884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:32.702845 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x5l2\" (UniqueName: \"kubernetes.io/projected/18303a78-30fe-4a3d-bbc6-d30bc4604c51-kube-api-access-4x5l2\") pod \"kserve-controller-manager-d6f4b65b8-5z45r\" (UID: \"18303a78-30fe-4a3d-bbc6-d30bc4604c51\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:33.195282 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:33.191200 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18303a78-30fe-4a3d-bbc6-d30bc4604c51-cert\") pod \"kserve-controller-manager-d6f4b65b8-5z45r\" (UID: \"18303a78-30fe-4a3d-bbc6-d30bc4604c51\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:33.195282 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:06:33.191477 2556 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Aug 04 16:06:33.195282 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:06:33.191565 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18303a78-30fe-4a3d-bbc6-d30bc4604c51-cert podName:18303a78-30fe-4a3d-bbc6-d30bc4604c51 nodeName:}" failed. No retries permitted until 2026-08-04 16:06:34.191542013 +0000 UTC m=+623.513965476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/18303a78-30fe-4a3d-bbc6-d30bc4604c51-cert") pod "kserve-controller-manager-d6f4b65b8-5z45r" (UID: "18303a78-30fe-4a3d-bbc6-d30bc4604c51") : secret "kserve-webhook-server-cert" not found Aug 04 16:06:34.201703 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:34.201656 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18303a78-30fe-4a3d-bbc6-d30bc4604c51-cert\") pod \"kserve-controller-manager-d6f4b65b8-5z45r\" (UID: \"18303a78-30fe-4a3d-bbc6-d30bc4604c51\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:34.204499 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:34.204472 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18303a78-30fe-4a3d-bbc6-d30bc4604c51-cert\") pod \"kserve-controller-manager-d6f4b65b8-5z45r\" (UID: \"18303a78-30fe-4a3d-bbc6-d30bc4604c51\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:34.315408 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:34.315362 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:34.783765 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:34.783593 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:06:34.783765 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:34.783693 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:06:34.783963 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:34.783772 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:06:34.949826 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:34.949790 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5z45r"] Aug 04 16:06:34.952094 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:06:34.952066 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18303a78_30fe_4a3d_bbc6_d30bc4604c51.slice/crio-fe814d5c37747ff3533dad35173078669fcb99fd3bef549ca2d35762e73f2c45 WatchSource:0}: Error finding container fe814d5c37747ff3533dad35173078669fcb99fd3bef549ca2d35762e73f2c45: Status 404 returned error can't find the container with id fe814d5c37747ff3533dad35173078669fcb99fd3bef549ca2d35762e73f2c45 Aug 04 16:06:35.747417 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:35.747382 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" event={"ID":"7a20b218-12f0-4e7b-ba34-579476907d4f","Type":"ContainerStarted","Data":"0db3844c34b5861b3874e9d0d67ba81419061c494e88f7c6059865993f7624eb"} Aug 04 16:06:35.747886 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:35.747569 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:06:35.749070 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:35.749045 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" event={"ID":"8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227","Type":"ContainerStarted","Data":"ff16c62b70fac936feafa578e7500db0a0cd44f61dc4f9146a00e93eae0c9ef9"} Aug 04 16:06:35.750208 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:35.750184 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" event={"ID":"18303a78-30fe-4a3d-bbc6-d30bc4604c51","Type":"ContainerStarted","Data":"fe814d5c37747ff3533dad35173078669fcb99fd3bef549ca2d35762e73f2c45"} Aug 04 16:06:35.788815 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:35.788749 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" podStartSLOduration=1.9784336439999999 podStartE2EDuration="6.788715004s" totalImagesPullingTime="4.81028136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:06:35.787581397 +0000 UTC m=+625.110004880" watchObservedRunningTime="2026-08-04 16:06:35.788715004 +0000 UTC m=+625.111138484" Aug 04 16:06:35.829342 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:35.829280 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" podStartSLOduration=1.915479016 podStartE2EDuration="6.829255837s" totalImagesPullingTime="4.913776821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:06:35.826916025 +0000 UTC m=+625.149339508" watchObservedRunningTime="2026-08-04 16:06:35.829255837 +0000 UTC m=+625.151679316" Aug 04 16:06:36.676899 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:36.676861 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:36.681586 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:36.681557 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:36.755716 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:36.755684 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:36.756670 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:36.756644 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k" Aug 04 16:06:40.778992 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:40.778950 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" event={"ID":"18303a78-30fe-4a3d-bbc6-d30bc4604c51","Type":"ContainerStarted","Data":"81cfb2204830afefb2b3614e8237f3cda4218ad14cde7594eb4f3e85405dedfe"} Aug 04 16:06:40.779415 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:40.779075 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:06:40.805874 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:40.805823 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" podStartSLOduration=4.036145193 podStartE2EDuration="8.805807169s" totalImagesPullingTime="4.769661976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:06:40.804226991 +0000 UTC m=+630.126650482" watchObservedRunningTime="2026-08-04 16:06:40.805807169 +0000 UTC m=+630.128230648" Aug 04 16:06:46.757779 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:06:46.757748 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5967759b96-mbcvc" Aug 04 16:07:11.789366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:11.789328 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5z45r" Aug 04 16:07:42.052468 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.052434 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs"] Aug 04 16:07:42.055831 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.055813 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:07:42.060308 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.060285 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 16:07:42.061229 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.061211 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vhhgd\"" Aug 04 16:07:42.061476 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.061458 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 16:07:42.083162 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.083130 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs"] Aug 04 16:07:42.108899 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.108864 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z88lq\" (UniqueName: \"kubernetes.io/projected/6dfb57d7-cffe-42fa-9c7a-a05544739edd-kube-api-access-z88lq\") pod \"kuadrant-operator-controller-manager-58fc56757c-rdqvs\" (UID: \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:07:42.109042 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.108914 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dfb57d7-cffe-42fa-9c7a-a05544739edd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-rdqvs\" (UID: \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:07:42.209845 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.209794 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z88lq\" (UniqueName: \"kubernetes.io/projected/6dfb57d7-cffe-42fa-9c7a-a05544739edd-kube-api-access-z88lq\") pod \"kuadrant-operator-controller-manager-58fc56757c-rdqvs\" (UID: \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:07:42.209845 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.209859 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dfb57d7-cffe-42fa-9c7a-a05544739edd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-rdqvs\" (UID: \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:07:42.210302 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.210277 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dfb57d7-cffe-42fa-9c7a-a05544739edd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-rdqvs\" (UID: \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:07:42.227483 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.227458 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z88lq\" (UniqueName: \"kubernetes.io/projected/6dfb57d7-cffe-42fa-9c7a-a05544739edd-kube-api-access-z88lq\") pod \"kuadrant-operator-controller-manager-58fc56757c-rdqvs\" (UID: \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:07:42.365670 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.365582 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:07:42.559509 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:42.559470 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs"] Aug 04 16:07:42.584827 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:07:42.584799 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dfb57d7_cffe_42fa_9c7a_a05544739edd.slice/crio-b7da31dbdbafdf3c82a82a98b07f89ff53229eff1bcbe18b2d6abb59050f070e WatchSource:0}: Error finding container b7da31dbdbafdf3c82a82a98b07f89ff53229eff1bcbe18b2d6abb59050f070e: Status 404 returned error can't find the container with id b7da31dbdbafdf3c82a82a98b07f89ff53229eff1bcbe18b2d6abb59050f070e Aug 04 16:07:43.055808 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:43.055772 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" event={"ID":"6dfb57d7-cffe-42fa-9c7a-a05544739edd","Type":"ContainerStarted","Data":"b7da31dbdbafdf3c82a82a98b07f89ff53229eff1bcbe18b2d6abb59050f070e"} Aug 04 16:07:48.372116 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:48.372074 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm"] Aug 04 16:07:48.376291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:48.376262 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" Aug 04 16:07:48.381790 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:48.381758 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-r8nz6\"" Aug 04 16:07:48.381984 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:48.381959 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Aug 04 16:07:48.393258 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:48.393229 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm"] Aug 04 16:07:48.489432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:48.489396 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmlxs\" (UniqueName: \"kubernetes.io/projected/7c736b8e-2fec-4380-ad41-e9c5ddd38939-kube-api-access-xmlxs\") pod \"dns-operator-controller-manager-5985564776-ml5nm\" (UID: \"7c736b8e-2fec-4380-ad41-e9c5ddd38939\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" Aug 04 16:07:48.590879 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:48.590836 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmlxs\" (UniqueName: \"kubernetes.io/projected/7c736b8e-2fec-4380-ad41-e9c5ddd38939-kube-api-access-xmlxs\") pod \"dns-operator-controller-manager-5985564776-ml5nm\" (UID: \"7c736b8e-2fec-4380-ad41-e9c5ddd38939\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" Aug 04 16:07:48.602213 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:48.602181 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmlxs\" (UniqueName: \"kubernetes.io/projected/7c736b8e-2fec-4380-ad41-e9c5ddd38939-kube-api-access-xmlxs\") pod \"dns-operator-controller-manager-5985564776-ml5nm\" (UID: \"7c736b8e-2fec-4380-ad41-e9c5ddd38939\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" Aug 04 16:07:48.690105 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:48.690017 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" Aug 04 16:07:49.534469 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:49.534443 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm"] Aug 04 16:07:49.538925 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:07:49.538900 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c736b8e_2fec_4380_ad41_e9c5ddd38939.slice/crio-91878e4b11f87fb3d36b0da8162bd71ea9e421bbb7ade5d541cde23da52e6c31 WatchSource:0}: Error finding container 91878e4b11f87fb3d36b0da8162bd71ea9e421bbb7ade5d541cde23da52e6c31: Status 404 returned error can't find the container with id 91878e4b11f87fb3d36b0da8162bd71ea9e421bbb7ade5d541cde23da52e6c31 Aug 04 16:07:50.092013 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:50.091975 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" event={"ID":"6dfb57d7-cffe-42fa-9c7a-a05544739edd","Type":"ContainerStarted","Data":"e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424"} Aug 04 16:07:50.092220 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:50.092096 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:07:50.093332 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:50.093310 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" event={"ID":"7c736b8e-2fec-4380-ad41-e9c5ddd38939","Type":"ContainerStarted","Data":"91878e4b11f87fb3d36b0da8162bd71ea9e421bbb7ade5d541cde23da52e6c31"} Aug 04 16:07:53.110889 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:53.110846 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" event={"ID":"7c736b8e-2fec-4380-ad41-e9c5ddd38939","Type":"ContainerStarted","Data":"f1feb73db46f6aae18aab08bbdd65ba73285d8a450163b9b97a410e981035b6f"} Aug 04 16:07:53.111331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:53.110930 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" Aug 04 16:07:53.145797 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:53.145744 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" podStartSLOduration=5.280130055 podStartE2EDuration="12.145706156s" totalImagesPullingTime="6.865576101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:07:50.134403138 +0000 UTC m=+699.456826621" watchObservedRunningTime="2026-08-04 16:07:53.145706156 +0000 UTC m=+702.468129636" Aug 04 16:07:53.146137 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:07:53.146117 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" podStartSLOduration=2.421478927 podStartE2EDuration="5.146109459s" totalImagesPullingTime="2.724630532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:07:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:07:53.144024998 +0000 UTC m=+702.466448478" watchObservedRunningTime="2026-08-04 16:07:53.146109459 +0000 UTC m=+702.468532938" Aug 04 16:08:01.100890 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:01.100854 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:08:03.087874 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.087835 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs"] Aug 04 16:08:03.088286 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.088060 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" podUID="6dfb57d7-cffe-42fa-9c7a-a05544739edd" containerName="manager" containerID="cri-o://e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424" gracePeriod=2 Aug 04 16:08:03.095768 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.095718 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs"] Aug 04 16:08:03.124672 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.124642 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz"] Aug 04 16:08:03.125146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.125122 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dfb57d7-cffe-42fa-9c7a-a05544739edd" containerName="manager" Aug 04 16:08:03.125146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.125146 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dfb57d7-cffe-42fa-9c7a-a05544739edd" containerName="manager" Aug 04 16:08:03.125337 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.125245 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dfb57d7-cffe-42fa-9c7a-a05544739edd" containerName="manager" Aug 04 16:08:03.130035 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.130014 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:03.132282 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:08:03.132253 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-rdqvs\" is forbidden: User \"system:node:ip-10-0-132-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-227.ec2.internal' and this object" podUID="6dfb57d7-cffe-42fa-9c7a-a05544739edd" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:08:03.138111 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.138087 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x"] Aug 04 16:08:03.141303 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.141273 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" Aug 04 16:08:03.144146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.144124 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-9s67t\"" Aug 04 16:08:03.144789 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.144718 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz"] Aug 04 16:08:03.189380 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.189345 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x"] Aug 04 16:08:03.193261 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:08:03.193220 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-rdqvs\" is forbidden: User \"system:node:ip-10-0-132-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-227.ec2.internal' and this object" podUID="6dfb57d7-cffe-42fa-9c7a-a05544739edd" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:08:03.226988 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.226951 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95fa2369-cade-4fb0-b241-b977290f312c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-hwdbz\" (UID: \"95fa2369-cade-4fb0-b241-b977290f312c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:03.227156 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.227016 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pzpv\" (UniqueName: \"kubernetes.io/projected/ea4f1663-cdd9-4299-8c6b-19917b0cb9aa-kube-api-access-7pzpv\") pod \"limitador-operator-controller-manager-6c549b6d85-68l6x\" (UID: \"ea4f1663-cdd9-4299-8c6b-19917b0cb9aa\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" Aug 04 16:08:03.227156 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.227054 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j9s2\" (UniqueName: \"kubernetes.io/projected/95fa2369-cade-4fb0-b241-b977290f312c-kube-api-access-5j9s2\") pod \"kuadrant-operator-controller-manager-58fc56757c-hwdbz\" (UID: \"95fa2369-cade-4fb0-b241-b977290f312c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:03.327927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.327893 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95fa2369-cade-4fb0-b241-b977290f312c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-hwdbz\" (UID: \"95fa2369-cade-4fb0-b241-b977290f312c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:03.328059 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.327996 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzpv\" (UniqueName: \"kubernetes.io/projected/ea4f1663-cdd9-4299-8c6b-19917b0cb9aa-kube-api-access-7pzpv\") pod \"limitador-operator-controller-manager-6c549b6d85-68l6x\" (UID: \"ea4f1663-cdd9-4299-8c6b-19917b0cb9aa\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" Aug 04 16:08:03.328059 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.328051 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j9s2\" (UniqueName: \"kubernetes.io/projected/95fa2369-cade-4fb0-b241-b977290f312c-kube-api-access-5j9s2\") pod \"kuadrant-operator-controller-manager-58fc56757c-hwdbz\" (UID: \"95fa2369-cade-4fb0-b241-b977290f312c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:03.328326 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.328303 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95fa2369-cade-4fb0-b241-b977290f312c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-hwdbz\" (UID: \"95fa2369-cade-4fb0-b241-b977290f312c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:03.337484 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.337461 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:08:03.340546 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.340487 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pzpv\" (UniqueName: \"kubernetes.io/projected/ea4f1663-cdd9-4299-8c6b-19917b0cb9aa-kube-api-access-7pzpv\") pod \"limitador-operator-controller-manager-6c549b6d85-68l6x\" (UID: \"ea4f1663-cdd9-4299-8c6b-19917b0cb9aa\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" Aug 04 16:08:03.341117 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.341093 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j9s2\" (UniqueName: \"kubernetes.io/projected/95fa2369-cade-4fb0-b241-b977290f312c-kube-api-access-5j9s2\") pod \"kuadrant-operator-controller-manager-58fc56757c-hwdbz\" (UID: \"95fa2369-cade-4fb0-b241-b977290f312c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:03.429298 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.429258 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z88lq\" (UniqueName: \"kubernetes.io/projected/6dfb57d7-cffe-42fa-9c7a-a05544739edd-kube-api-access-z88lq\") pod \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\" (UID: \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\") " Aug 04 16:08:03.429298 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.429310 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dfb57d7-cffe-42fa-9c7a-a05544739edd-extensions-socket-volume\") pod \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\" (UID: \"6dfb57d7-cffe-42fa-9c7a-a05544739edd\") " Aug 04 16:08:03.429866 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.429838 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dfb57d7-cffe-42fa-9c7a-a05544739edd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6dfb57d7-cffe-42fa-9c7a-a05544739edd" (UID: "6dfb57d7-cffe-42fa-9c7a-a05544739edd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:08:03.431521 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.431494 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dfb57d7-cffe-42fa-9c7a-a05544739edd-kube-api-access-z88lq" (OuterVolumeSpecName: "kube-api-access-z88lq") pod "6dfb57d7-cffe-42fa-9c7a-a05544739edd" (UID: "6dfb57d7-cffe-42fa-9c7a-a05544739edd"). InnerVolumeSpecName "kube-api-access-z88lq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:08:03.473290 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.473257 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:03.479442 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.479420 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" Aug 04 16:08:03.530800 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.530716 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z88lq\" (UniqueName: \"kubernetes.io/projected/6dfb57d7-cffe-42fa-9c7a-a05544739edd-kube-api-access-z88lq\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:08:03.530800 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.530772 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dfb57d7-cffe-42fa-9c7a-a05544739edd-extensions-socket-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:08:03.628369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.628344 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz"] Aug 04 16:08:03.631103 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:08:03.631077 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95fa2369_cade_4fb0_b241_b977290f312c.slice/crio-bf21020c0d3ae34c35975ed1eb1b4d752f65afa0265b4aea0abb5b7ab2f98a5c WatchSource:0}: Error finding container bf21020c0d3ae34c35975ed1eb1b4d752f65afa0265b4aea0abb5b7ab2f98a5c: Status 404 returned error can't find the container with id bf21020c0d3ae34c35975ed1eb1b4d752f65afa0265b4aea0abb5b7ab2f98a5c Aug 04 16:08:03.659193 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:03.659168 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x"] Aug 04 16:08:03.674920 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:08:03.674889 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4f1663_cdd9_4299_8c6b_19917b0cb9aa.slice/crio-bf310f8711f8ab67002879b2bc48fdf6e52c29d5baf86dcbedd7cd098249cf9d WatchSource:0}: Error finding container bf310f8711f8ab67002879b2bc48fdf6e52c29d5baf86dcbedd7cd098249cf9d: Status 404 returned error can't find the container with id bf310f8711f8ab67002879b2bc48fdf6e52c29d5baf86dcbedd7cd098249cf9d Aug 04 16:08:04.117876 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.117847 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ml5nm" Aug 04 16:08:04.164799 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.164757 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" event={"ID":"ea4f1663-cdd9-4299-8c6b-19917b0cb9aa","Type":"ContainerStarted","Data":"bf310f8711f8ab67002879b2bc48fdf6e52c29d5baf86dcbedd7cd098249cf9d"} Aug 04 16:08:04.166695 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.166663 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" event={"ID":"95fa2369-cade-4fb0-b241-b977290f312c","Type":"ContainerStarted","Data":"a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b"} Aug 04 16:08:04.166886 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.166742 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" event={"ID":"95fa2369-cade-4fb0-b241-b977290f312c","Type":"ContainerStarted","Data":"bf21020c0d3ae34c35975ed1eb1b4d752f65afa0265b4aea0abb5b7ab2f98a5c"} Aug 04 16:08:04.166886 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.166878 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:04.168310 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.168288 2556 generic.go:356] "Generic (PLEG): container finished" podID="6dfb57d7-cffe-42fa-9c7a-a05544739edd" containerID="e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424" exitCode=0 Aug 04 16:08:04.168422 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.168361 2556 scope.go:117] "RemoveContainer" containerID="e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424" Aug 04 16:08:04.168535 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.168514 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rdqvs" Aug 04 16:08:04.181082 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.180701 2556 scope.go:117] "RemoveContainer" containerID="e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424" Aug 04 16:08:04.181357 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:08:04.181331 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424\": container with ID starting with e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424 not found: ID does not exist" containerID="e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424" Aug 04 16:08:04.181439 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.181372 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424"} err="failed to get container status \"e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424\": rpc error: code = NotFound desc = could not find container \"e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424\": container with ID starting with e3a2bf6aaa06ed1064fa9955f40a208feceaf6e555f5f42e578ad00a53c09424 not found: ID does not exist" Aug 04 16:08:04.207201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:04.207139 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" podStartSLOduration=1.207117776 podStartE2EDuration="1.207117776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:08:04.202317436 +0000 UTC m=+713.524740916" watchObservedRunningTime="2026-08-04 16:08:04.207117776 +0000 UTC m=+713.529541257" Aug 04 16:08:05.174938 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:05.174895 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" event={"ID":"ea4f1663-cdd9-4299-8c6b-19917b0cb9aa","Type":"ContainerStarted","Data":"02152fafa983712f9304bae6086aae148cd944646e020a9498d1d69f2c846b22"} Aug 04 16:08:05.175381 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:05.174972 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" Aug 04 16:08:05.204666 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:05.204609 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" podStartSLOduration=0.923493925 podStartE2EDuration="2.204593042s" totalImagesPullingTime="1.281099117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:08:05.203275547 +0000 UTC m=+714.525699028" watchObservedRunningTime="2026-08-04 16:08:05.204593042 +0000 UTC m=+714.527016522" Aug 04 16:08:05.279217 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:05.279184 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dfb57d7-cffe-42fa-9c7a-a05544739edd" path="/var/lib/kubelet/pods/6dfb57d7-cffe-42fa-9c7a-a05544739edd/volumes" Aug 04 16:08:15.178834 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:15.178799 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:16.183191 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:16.183161 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-68l6x" Aug 04 16:08:20.547920 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:20.547882 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz"] Aug 04 16:08:20.548329 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:20.548093 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" podUID="95fa2369-cade-4fb0-b241-b977290f312c" containerName="manager" containerID="cri-o://a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b" gracePeriod=10 Aug 04 16:08:21.091954 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.091930 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:21.177757 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.177634 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j9s2\" (UniqueName: \"kubernetes.io/projected/95fa2369-cade-4fb0-b241-b977290f312c-kube-api-access-5j9s2\") pod \"95fa2369-cade-4fb0-b241-b977290f312c\" (UID: \"95fa2369-cade-4fb0-b241-b977290f312c\") " Aug 04 16:08:21.177757 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.177683 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95fa2369-cade-4fb0-b241-b977290f312c-extensions-socket-volume\") pod \"95fa2369-cade-4fb0-b241-b977290f312c\" (UID: \"95fa2369-cade-4fb0-b241-b977290f312c\") " Aug 04 16:08:21.178116 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.178089 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95fa2369-cade-4fb0-b241-b977290f312c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "95fa2369-cade-4fb0-b241-b977290f312c" (UID: "95fa2369-cade-4fb0-b241-b977290f312c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:08:21.179774 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.179748 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95fa2369-cade-4fb0-b241-b977290f312c-kube-api-access-5j9s2" (OuterVolumeSpecName: "kube-api-access-5j9s2") pod "95fa2369-cade-4fb0-b241-b977290f312c" (UID: "95fa2369-cade-4fb0-b241-b977290f312c"). InnerVolumeSpecName "kube-api-access-5j9s2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:08:21.257962 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.257922 2556 generic.go:356] "Generic (PLEG): container finished" podID="95fa2369-cade-4fb0-b241-b977290f312c" containerID="a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b" exitCode=0 Aug 04 16:08:21.257962 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.257965 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" Aug 04 16:08:21.258156 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.258012 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" event={"ID":"95fa2369-cade-4fb0-b241-b977290f312c","Type":"ContainerDied","Data":"a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b"} Aug 04 16:08:21.258156 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.258063 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz" event={"ID":"95fa2369-cade-4fb0-b241-b977290f312c","Type":"ContainerDied","Data":"bf21020c0d3ae34c35975ed1eb1b4d752f65afa0265b4aea0abb5b7ab2f98a5c"} Aug 04 16:08:21.258156 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.258087 2556 scope.go:117] "RemoveContainer" containerID="a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b" Aug 04 16:08:21.266875 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.266852 2556 scope.go:117] "RemoveContainer" containerID="a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b" Aug 04 16:08:21.267200 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:08:21.267181 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b\": container with ID starting with a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b not found: ID does not exist" containerID="a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b" Aug 04 16:08:21.267250 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.267209 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b"} err="failed to get container status \"a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b\": rpc error: code = NotFound desc = could not find container \"a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b\": container with ID starting with a6eaee0c383a8ef33ec545c146c38e1f0af64c13c177e3f7838c68127b61510b not found: ID does not exist" Aug 04 16:08:21.278472 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.278446 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j9s2\" (UniqueName: \"kubernetes.io/projected/95fa2369-cade-4fb0-b241-b977290f312c-kube-api-access-5j9s2\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:08:21.278472 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.278467 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95fa2369-cade-4fb0-b241-b977290f312c-extensions-socket-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:08:21.284019 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.283991 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz"] Aug 04 16:08:21.286668 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:21.286645 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-hwdbz"] Aug 04 16:08:23.272467 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:23.272423 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95fa2369-cade-4fb0-b241-b977290f312c" path="/var/lib/kubelet/pods/95fa2369-cade-4fb0-b241-b977290f312c/volumes" Aug 04 16:08:36.918392 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:36.918355 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr"] Aug 04 16:08:36.919402 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:36.919368 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95fa2369-cade-4fb0-b241-b977290f312c" containerName="manager" Aug 04 16:08:36.919402 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:36.919400 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="95fa2369-cade-4fb0-b241-b977290f312c" containerName="manager" Aug 04 16:08:36.919540 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:36.919519 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="95fa2369-cade-4fb0-b241-b977290f312c" containerName="manager" Aug 04 16:08:36.924031 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:36.924007 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:36.926787 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:36.926764 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pbz7w\"" Aug 04 16:08:36.937807 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:36.937739 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr"] Aug 04 16:08:37.014112 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.014074 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.014286 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.014124 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.014286 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.014158 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.014286 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.014203 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9jlt\" (UniqueName: \"kubernetes.io/projected/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-kube-api-access-c9jlt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.014406 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.014294 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.014406 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.014335 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.014406 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.014358 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.014406 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.014374 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.014406 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.014398 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.115459 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.115410 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jlt\" (UniqueName: \"kubernetes.io/projected/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-kube-api-access-c9jlt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.115648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.115473 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.115648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.115500 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.115648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.115521 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.115648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.115541 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.115648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.115562 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.115648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.115597 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.115648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.115624 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.116056 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.115655 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.116056 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.116001 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.116056 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.116021 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.116273 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.116243 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.116443 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.116426 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.116603 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.116578 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.118053 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.118027 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.118134 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.118080 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.124383 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.124350 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.124477 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.124447 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9jlt\" (UniqueName: \"kubernetes.io/projected/9f3fb966-8f4b-4709-bbbf-1f45529c4eaf-kube-api-access-c9jlt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7gqdr\" (UID: \"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.239623 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.239590 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:37.395471 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.395444 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr"] Aug 04 16:08:37.398061 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:08:37.398032 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f3fb966_8f4b_4709_bbbf_1f45529c4eaf.slice/crio-2f788a322f07e83db17a8b3a880719b8133616256f56102f7c749a01ec89774e WatchSource:0}: Error finding container 2f788a322f07e83db17a8b3a880719b8133616256f56102f7c749a01ec89774e: Status 404 returned error can't find the container with id 2f788a322f07e83db17a8b3a880719b8133616256f56102f7c749a01ec89774e Aug 04 16:08:37.401213 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.401169 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:08:37.401287 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.401260 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:08:37.401324 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:37.401299 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:08:38.339238 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:38.339202 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" event={"ID":"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf","Type":"ContainerStarted","Data":"6cf91f21ba9ee01d2fc30874017c9115a37facbfc36ed0940edaa33b73ae8934"} Aug 04 16:08:38.339658 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:38.339259 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" event={"ID":"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf","Type":"ContainerStarted","Data":"2f788a322f07e83db17a8b3a880719b8133616256f56102f7c749a01ec89774e"} Aug 04 16:08:38.380353 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:38.380298 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" podStartSLOduration=2.3802793380000002 podStartE2EDuration="2.380279338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:08:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:08:38.379069393 +0000 UTC m=+747.701492875" watchObservedRunningTime="2026-08-04 16:08:38.380279338 +0000 UTC m=+747.702702818" Aug 04 16:08:39.240515 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.240470 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:39.245499 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.245470 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:39.344231 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.344201 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:39.351073 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.351043 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:08:39.491593 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.491508 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-dfwmt"] Aug 04 16:08:39.495803 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.495779 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:39.498310 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.498285 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Aug 04 16:08:39.498310 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.498294 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vk9xc\"" Aug 04 16:08:39.503872 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.503847 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-dfwmt"] Aug 04 16:08:39.593178 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.593143 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-dfwmt"] Aug 04 16:08:39.639450 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.639414 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gbsk\" (UniqueName: \"kubernetes.io/projected/662999e6-c997-4858-ac46-aba766cfa644-kube-api-access-4gbsk\") pod \"limitador-limitador-859c4d96d6-dfwmt\" (UID: \"662999e6-c997-4858-ac46-aba766cfa644\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:39.639628 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.639473 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/662999e6-c997-4858-ac46-aba766cfa644-config-file\") pod \"limitador-limitador-859c4d96d6-dfwmt\" (UID: \"662999e6-c997-4858-ac46-aba766cfa644\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:39.740572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.740531 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/662999e6-c997-4858-ac46-aba766cfa644-config-file\") pod \"limitador-limitador-859c4d96d6-dfwmt\" (UID: \"662999e6-c997-4858-ac46-aba766cfa644\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:39.740805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.740631 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gbsk\" (UniqueName: \"kubernetes.io/projected/662999e6-c997-4858-ac46-aba766cfa644-kube-api-access-4gbsk\") pod \"limitador-limitador-859c4d96d6-dfwmt\" (UID: \"662999e6-c997-4858-ac46-aba766cfa644\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:39.741275 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.741251 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/662999e6-c997-4858-ac46-aba766cfa644-config-file\") pod \"limitador-limitador-859c4d96d6-dfwmt\" (UID: \"662999e6-c997-4858-ac46-aba766cfa644\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:39.750503 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.750436 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gbsk\" (UniqueName: \"kubernetes.io/projected/662999e6-c997-4858-ac46-aba766cfa644-kube-api-access-4gbsk\") pod \"limitador-limitador-859c4d96d6-dfwmt\" (UID: \"662999e6-c997-4858-ac46-aba766cfa644\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:39.807548 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.807506 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:39.946567 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:39.946541 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-dfwmt"] Aug 04 16:08:39.948123 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:08:39.948084 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod662999e6_c997_4858_ac46_aba766cfa644.slice/crio-f580ac320952b13357955a2d0dae5fc1835bf5d959abb24a9b46e7a9a126d5ad WatchSource:0}: Error finding container f580ac320952b13357955a2d0dae5fc1835bf5d959abb24a9b46e7a9a126d5ad: Status 404 returned error can't find the container with id f580ac320952b13357955a2d0dae5fc1835bf5d959abb24a9b46e7a9a126d5ad Aug 04 16:08:40.350056 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:40.350011 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" event={"ID":"662999e6-c997-4858-ac46-aba766cfa644","Type":"ContainerStarted","Data":"f580ac320952b13357955a2d0dae5fc1835bf5d959abb24a9b46e7a9a126d5ad"} Aug 04 16:08:42.361676 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:42.361639 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" event={"ID":"662999e6-c997-4858-ac46-aba766cfa644","Type":"ContainerStarted","Data":"b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be"} Aug 04 16:08:42.362140 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:42.361774 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:42.381664 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:42.381608 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" podStartSLOduration=1.902603558 podStartE2EDuration="3.381592112s" totalImagesPullingTime="1.478988554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:08:42.380459125 +0000 UTC m=+751.702882614" watchObservedRunningTime="2026-08-04 16:08:42.381592112 +0000 UTC m=+751.704015591" Aug 04 16:08:52.714321 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:52.714282 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-dfwmt"] Aug 04 16:08:52.714803 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:52.714496 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" podUID="662999e6-c997-4858-ac46-aba766cfa644" containerName="limitador" containerID="cri-o://b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be" gracePeriod=30 Aug 04 16:08:52.715259 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:52.715240 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:53.275120 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.275052 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:53.366053 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.366018 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/662999e6-c997-4858-ac46-aba766cfa644-config-file\") pod \"662999e6-c997-4858-ac46-aba766cfa644\" (UID: \"662999e6-c997-4858-ac46-aba766cfa644\") " Aug 04 16:08:53.366221 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.366105 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gbsk\" (UniqueName: \"kubernetes.io/projected/662999e6-c997-4858-ac46-aba766cfa644-kube-api-access-4gbsk\") pod \"662999e6-c997-4858-ac46-aba766cfa644\" (UID: \"662999e6-c997-4858-ac46-aba766cfa644\") " Aug 04 16:08:53.366395 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.366363 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/662999e6-c997-4858-ac46-aba766cfa644-config-file" (OuterVolumeSpecName: "config-file") pod "662999e6-c997-4858-ac46-aba766cfa644" (UID: "662999e6-c997-4858-ac46-aba766cfa644"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:08:53.368171 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.368146 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/662999e6-c997-4858-ac46-aba766cfa644-kube-api-access-4gbsk" (OuterVolumeSpecName: "kube-api-access-4gbsk") pod "662999e6-c997-4858-ac46-aba766cfa644" (UID: "662999e6-c997-4858-ac46-aba766cfa644"). InnerVolumeSpecName "kube-api-access-4gbsk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:08:53.420785 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.420719 2556 generic.go:356] "Generic (PLEG): container finished" podID="662999e6-c997-4858-ac46-aba766cfa644" containerID="b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be" exitCode=0 Aug 04 16:08:53.420785 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.420786 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" Aug 04 16:08:53.420983 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.420817 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" event={"ID":"662999e6-c997-4858-ac46-aba766cfa644","Type":"ContainerDied","Data":"b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be"} Aug 04 16:08:53.420983 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.420878 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-dfwmt" event={"ID":"662999e6-c997-4858-ac46-aba766cfa644","Type":"ContainerDied","Data":"f580ac320952b13357955a2d0dae5fc1835bf5d959abb24a9b46e7a9a126d5ad"} Aug 04 16:08:53.420983 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.420904 2556 scope.go:117] "RemoveContainer" containerID="b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be" Aug 04 16:08:53.430008 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.429987 2556 scope.go:117] "RemoveContainer" containerID="b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be" Aug 04 16:08:53.430277 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:08:53.430254 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be\": container with ID starting with b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be not found: ID does not exist" containerID="b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be" Aug 04 16:08:53.430367 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.430285 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be"} err="failed to get container status \"b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be\": rpc error: code = NotFound desc = could not find container \"b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be\": container with ID starting with b1bfbf5d283ed6846a222832a9a3868f70ef583b0df65113888c236f6c1a14be not found: ID does not exist" Aug 04 16:08:53.444269 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.444231 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-dfwmt"] Aug 04 16:08:53.448285 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.448262 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-dfwmt"] Aug 04 16:08:53.467066 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.467040 2556 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/662999e6-c997-4858-ac46-aba766cfa644-config-file\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:08:53.467066 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:53.467062 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gbsk\" (UniqueName: \"kubernetes.io/projected/662999e6-c997-4858-ac46-aba766cfa644-kube-api-access-4gbsk\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:08:55.273893 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:55.273852 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="662999e6-c997-4858-ac46-aba766cfa644" path="/var/lib/kubelet/pods/662999e6-c997-4858-ac46-aba766cfa644/volumes" Aug 04 16:08:55.992103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:55.992069 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8"] Aug 04 16:08:55.992453 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:55.992436 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="662999e6-c997-4858-ac46-aba766cfa644" containerName="limitador" Aug 04 16:08:55.992496 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:55.992455 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="662999e6-c997-4858-ac46-aba766cfa644" containerName="limitador" Aug 04 16:08:55.992550 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:55.992540 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="662999e6-c997-4858-ac46-aba766cfa644" containerName="limitador" Aug 04 16:08:55.997393 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:55.997366 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:08:56.001306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.001252 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 04 16:08:56.001868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.001851 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Aug 04 16:08:56.002197 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.002172 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 16:08:56.002481 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.002444 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cmm6g\"" Aug 04 16:08:56.006052 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.006010 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8"] Aug 04 16:08:56.091453 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.091413 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8dx9\" (UniqueName: \"kubernetes.io/projected/32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67-kube-api-access-z8dx9\") pod \"postgres-69fb4594cf-rbzc8\" (UID: \"32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:08:56.091651 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.091623 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67-data\") pod \"postgres-69fb4594cf-rbzc8\" (UID: \"32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:08:56.192710 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.192676 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8dx9\" (UniqueName: \"kubernetes.io/projected/32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67-kube-api-access-z8dx9\") pod \"postgres-69fb4594cf-rbzc8\" (UID: \"32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:08:56.192893 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.192771 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67-data\") pod \"postgres-69fb4594cf-rbzc8\" (UID: \"32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:08:56.193164 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.193143 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67-data\") pod \"postgres-69fb4594cf-rbzc8\" (UID: \"32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:08:56.209642 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.209610 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8dx9\" (UniqueName: \"kubernetes.io/projected/32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67-kube-api-access-z8dx9\") pod \"postgres-69fb4594cf-rbzc8\" (UID: \"32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:08:56.315943 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.315844 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:08:56.450761 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:56.450713 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8"] Aug 04 16:08:56.452537 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:08:56.452515 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32de1bf2_b2b9_4eb8_8ca5_db7fc4c01d67.slice/crio-e05887539618133079788bf002539f93fea536b6d1d6e397335666b5cd22a457 WatchSource:0}: Error finding container e05887539618133079788bf002539f93fea536b6d1d6e397335666b5cd22a457: Status 404 returned error can't find the container with id e05887539618133079788bf002539f93fea536b6d1d6e397335666b5cd22a457 Aug 04 16:08:57.446540 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:08:57.446494 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" event={"ID":"32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67","Type":"ContainerStarted","Data":"e05887539618133079788bf002539f93fea536b6d1d6e397335666b5cd22a457"} Aug 04 16:09:01.471472 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:01.471438 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" event={"ID":"32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67","Type":"ContainerStarted","Data":"6cc645885069d9d701ef8270e572f27f47a0b6f916edccc7e093f8d120699a81"} Aug 04 16:09:01.471846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:01.471527 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:09:01.496532 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:01.496479 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" podStartSLOduration=1.629217261 podStartE2EDuration="6.496462535s" totalImagesPullingTime="4.867245274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:09:01.494446895 +0000 UTC m=+770.816870376" watchObservedRunningTime="2026-08-04 16:09:01.496462535 +0000 UTC m=+770.818886014" Aug 04 16:09:07.507522 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:07.507492 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rbzc8" Aug 04 16:09:08.343973 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.343939 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-cjr9m"] Aug 04 16:09:08.350249 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.350224 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:08.356477 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.356453 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Aug 04 16:09:08.356918 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.356635 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rv57c\"" Aug 04 16:09:08.362295 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.362273 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-cjr9m"] Aug 04 16:09:08.403927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.403887 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a99413c-d9f8-493a-9834-290580456232-tls-cert\") pod \"authorino-687bd45585-cjr9m\" (UID: \"6a99413c-d9f8-493a-9834-290580456232\") " pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:08.404094 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.404023 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l8wc\" (UniqueName: \"kubernetes.io/projected/6a99413c-d9f8-493a-9834-290580456232-kube-api-access-7l8wc\") pod \"authorino-687bd45585-cjr9m\" (UID: \"6a99413c-d9f8-493a-9834-290580456232\") " pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:08.504501 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.504449 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l8wc\" (UniqueName: \"kubernetes.io/projected/6a99413c-d9f8-493a-9834-290580456232-kube-api-access-7l8wc\") pod \"authorino-687bd45585-cjr9m\" (UID: \"6a99413c-d9f8-493a-9834-290580456232\") " pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:08.504501 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.504506 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a99413c-d9f8-493a-9834-290580456232-tls-cert\") pod \"authorino-687bd45585-cjr9m\" (UID: \"6a99413c-d9f8-493a-9834-290580456232\") " pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:08.507385 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.507353 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a99413c-d9f8-493a-9834-290580456232-tls-cert\") pod \"authorino-687bd45585-cjr9m\" (UID: \"6a99413c-d9f8-493a-9834-290580456232\") " pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:08.528857 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.528831 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l8wc\" (UniqueName: \"kubernetes.io/projected/6a99413c-d9f8-493a-9834-290580456232-kube-api-access-7l8wc\") pod \"authorino-687bd45585-cjr9m\" (UID: \"6a99413c-d9f8-493a-9834-290580456232\") " pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:08.530396 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.530376 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-cjr9m"] Aug 04 16:09:08.530648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.530635 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:08.564544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.564514 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75b9569c58-zm8r7"] Aug 04 16:09:08.568823 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.568800 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:09:08.583287 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.583256 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75b9569c58-zm8r7"] Aug 04 16:09:08.605153 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.605065 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq2dr\" (UniqueName: \"kubernetes.io/projected/d70f1e47-c0c4-4460-a485-0f7eac5a4114-kube-api-access-wq2dr\") pod \"authorino-75b9569c58-zm8r7\" (UID: \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\") " pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:09:08.605153 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.605120 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d70f1e47-c0c4-4460-a485-0f7eac5a4114-tls-cert\") pod \"authorino-75b9569c58-zm8r7\" (UID: \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\") " pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:09:08.673225 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.673199 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-cjr9m"] Aug 04 16:09:08.675302 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:09:08.675275 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a99413c_d9f8_493a_9834_290580456232.slice/crio-55e8140f60925e09af5bdda3c27b5d7f83e032c104d3307642bcd58ee741c3fd WatchSource:0}: Error finding container 55e8140f60925e09af5bdda3c27b5d7f83e032c104d3307642bcd58ee741c3fd: Status 404 returned error can't find the container with id 55e8140f60925e09af5bdda3c27b5d7f83e032c104d3307642bcd58ee741c3fd Aug 04 16:09:08.706274 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.706236 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2dr\" (UniqueName: \"kubernetes.io/projected/d70f1e47-c0c4-4460-a485-0f7eac5a4114-kube-api-access-wq2dr\") pod \"authorino-75b9569c58-zm8r7\" (UID: \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\") " pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:09:08.706387 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.706290 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d70f1e47-c0c4-4460-a485-0f7eac5a4114-tls-cert\") pod \"authorino-75b9569c58-zm8r7\" (UID: \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\") " pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:09:08.708777 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.708752 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d70f1e47-c0c4-4460-a485-0f7eac5a4114-tls-cert\") pod \"authorino-75b9569c58-zm8r7\" (UID: \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\") " pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:09:08.716956 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.716938 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq2dr\" (UniqueName: \"kubernetes.io/projected/d70f1e47-c0c4-4460-a485-0f7eac5a4114-kube-api-access-wq2dr\") pod \"authorino-75b9569c58-zm8r7\" (UID: \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\") " pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:09:08.879454 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:08.879374 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:09:09.032373 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:09.032347 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75b9569c58-zm8r7"] Aug 04 16:09:09.034182 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:09:09.034153 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd70f1e47_c0c4_4460_a485_0f7eac5a4114.slice/crio-afc164087cb8ea91ead80fdd50c44d4ff847fc32c9ce4bb486ebf79836065728 WatchSource:0}: Error finding container afc164087cb8ea91ead80fdd50c44d4ff847fc32c9ce4bb486ebf79836065728: Status 404 returned error can't find the container with id afc164087cb8ea91ead80fdd50c44d4ff847fc32c9ce4bb486ebf79836065728 Aug 04 16:09:09.519313 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:09.519245 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687bd45585-cjr9m" event={"ID":"6a99413c-d9f8-493a-9834-290580456232","Type":"ContainerStarted","Data":"55e8140f60925e09af5bdda3c27b5d7f83e032c104d3307642bcd58ee741c3fd"} Aug 04 16:09:09.520931 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:09.520892 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b9569c58-zm8r7" event={"ID":"d70f1e47-c0c4-4460-a485-0f7eac5a4114","Type":"ContainerStarted","Data":"afc164087cb8ea91ead80fdd50c44d4ff847fc32c9ce4bb486ebf79836065728"} Aug 04 16:09:11.537990 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.537886 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687bd45585-cjr9m" event={"ID":"6a99413c-d9f8-493a-9834-290580456232","Type":"ContainerStarted","Data":"f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f"} Aug 04 16:09:11.537990 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.537945 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-687bd45585-cjr9m" podUID="6a99413c-d9f8-493a-9834-290580456232" containerName="authorino" containerID="cri-o://f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f" gracePeriod=30 Aug 04 16:09:11.539405 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.539373 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b9569c58-zm8r7" event={"ID":"d70f1e47-c0c4-4460-a485-0f7eac5a4114","Type":"ContainerStarted","Data":"692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a"} Aug 04 16:09:11.560303 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.560262 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-687bd45585-cjr9m" podStartSLOduration=1.369612337 podStartE2EDuration="3.560248571s" totalImagesPullingTime="2.190636234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:09:11.558231998 +0000 UTC m=+780.880655478" watchObservedRunningTime="2026-08-04 16:09:11.560248571 +0000 UTC m=+780.882672051" Aug 04 16:09:11.584374 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.584334 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75b9569c58-zm8r7" podStartSLOduration=1.744240827 podStartE2EDuration="3.584320551s" totalImagesPullingTime="1.840079724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:09:11.583579032 +0000 UTC m=+780.906002513" watchObservedRunningTime="2026-08-04 16:09:11.584320551 +0000 UTC m=+780.906744031" Aug 04 16:09:11.781311 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.781279 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:11.833457 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.833382 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a99413c-d9f8-493a-9834-290580456232-tls-cert\") pod \"6a99413c-d9f8-493a-9834-290580456232\" (UID: \"6a99413c-d9f8-493a-9834-290580456232\") " Aug 04 16:09:11.833608 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.833488 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l8wc\" (UniqueName: \"kubernetes.io/projected/6a99413c-d9f8-493a-9834-290580456232-kube-api-access-7l8wc\") pod \"6a99413c-d9f8-493a-9834-290580456232\" (UID: \"6a99413c-d9f8-493a-9834-290580456232\") " Aug 04 16:09:11.835379 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.835350 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a99413c-d9f8-493a-9834-290580456232-kube-api-access-7l8wc" (OuterVolumeSpecName: "kube-api-access-7l8wc") pod "6a99413c-d9f8-493a-9834-290580456232" (UID: "6a99413c-d9f8-493a-9834-290580456232"). InnerVolumeSpecName "kube-api-access-7l8wc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:09:11.843419 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.843395 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a99413c-d9f8-493a-9834-290580456232-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6a99413c-d9f8-493a-9834-290580456232" (UID: "6a99413c-d9f8-493a-9834-290580456232"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:09:11.934905 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.934866 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l8wc\" (UniqueName: \"kubernetes.io/projected/6a99413c-d9f8-493a-9834-290580456232-kube-api-access-7l8wc\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:09:11.934905 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:11.934896 2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a99413c-d9f8-493a-9834-290580456232-tls-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:09:12.545838 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:12.545795 2556 generic.go:356] "Generic (PLEG): container finished" podID="6a99413c-d9f8-493a-9834-290580456232" containerID="f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f" exitCode=0 Aug 04 16:09:12.545838 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:12.545841 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-cjr9m" Aug 04 16:09:12.546327 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:12.545872 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687bd45585-cjr9m" event={"ID":"6a99413c-d9f8-493a-9834-290580456232","Type":"ContainerDied","Data":"f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f"} Aug 04 16:09:12.546327 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:12.545928 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687bd45585-cjr9m" event={"ID":"6a99413c-d9f8-493a-9834-290580456232","Type":"ContainerDied","Data":"55e8140f60925e09af5bdda3c27b5d7f83e032c104d3307642bcd58ee741c3fd"} Aug 04 16:09:12.546327 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:12.545956 2556 scope.go:117] "RemoveContainer" containerID="f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f" Aug 04 16:09:12.554775 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:12.554755 2556 scope.go:117] "RemoveContainer" containerID="f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f" Aug 04 16:09:12.555013 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:09:12.554991 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f\": container with ID starting with f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f not found: ID does not exist" containerID="f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f" Aug 04 16:09:12.555087 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:12.555025 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f"} err="failed to get container status \"f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f\": rpc error: code = NotFound desc = could not find container \"f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f\": container with ID starting with f7922f88d19db8906d78a670a4a39f6bee8a19946917cb3fff217a9a9567a18f not found: ID does not exist" Aug 04 16:09:12.568048 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:12.568026 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-cjr9m"] Aug 04 16:09:12.572429 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:12.572409 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-cjr9m"] Aug 04 16:09:13.274168 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:13.274129 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a99413c-d9f8-493a-9834-290580456232" path="/var/lib/kubelet/pods/6a99413c-d9f8-493a-9834-290580456232/volumes" Aug 04 16:09:16.531085 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.531049 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-974d9c7f7-jtmrf"] Aug 04 16:09:16.531487 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.531438 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a99413c-d9f8-493a-9834-290580456232" containerName="authorino" Aug 04 16:09:16.531487 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.531450 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a99413c-d9f8-493a-9834-290580456232" containerName="authorino" Aug 04 16:09:16.531602 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.531531 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a99413c-d9f8-493a-9834-290580456232" containerName="authorino" Aug 04 16:09:16.555367 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.555329 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-974d9c7f7-jtmrf"] Aug 04 16:09:16.555515 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.555431 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:16.558224 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.558193 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 16:09:16.558434 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.558404 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-42s8l\"" Aug 04 16:09:16.558554 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.558252 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 16:09:16.577580 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.577551 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-webhook-cert\") pod \"maas-controller-974d9c7f7-jtmrf\" (UID: \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\") " pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:16.577736 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.577600 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsk25\" (UniqueName: \"kubernetes.io/projected/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-kube-api-access-fsk25\") pod \"maas-controller-974d9c7f7-jtmrf\" (UID: \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\") " pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:16.678355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.678316 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-webhook-cert\") pod \"maas-controller-974d9c7f7-jtmrf\" (UID: \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\") " pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:16.678355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.678358 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsk25\" (UniqueName: \"kubernetes.io/projected/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-kube-api-access-fsk25\") pod \"maas-controller-974d9c7f7-jtmrf\" (UID: \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\") " pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:16.680978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.680953 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-webhook-cert\") pod \"maas-controller-974d9c7f7-jtmrf\" (UID: \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\") " pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:16.688012 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.687984 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsk25\" (UniqueName: \"kubernetes.io/projected/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-kube-api-access-fsk25\") pod \"maas-controller-974d9c7f7-jtmrf\" (UID: \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\") " pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:16.868793 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:16.868654 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:17.010058 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:17.010035 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-974d9c7f7-jtmrf"] Aug 04 16:09:17.011975 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:09:17.011939 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8bc7dd_b0e1_44ce_9afd_a406d256c93d.slice/crio-41ae88f0d5babd5b13f1ec9256b3f0ee814c1ace2c7c414441b9d896c487e7a5 WatchSource:0}: Error finding container 41ae88f0d5babd5b13f1ec9256b3f0ee814c1ace2c7c414441b9d896c487e7a5: Status 404 returned error can't find the container with id 41ae88f0d5babd5b13f1ec9256b3f0ee814c1ace2c7c414441b9d896c487e7a5 Aug 04 16:09:17.571749 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:17.571687 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" event={"ID":"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d","Type":"ContainerStarted","Data":"41ae88f0d5babd5b13f1ec9256b3f0ee814c1ace2c7c414441b9d896c487e7a5"} Aug 04 16:09:20.589903 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:20.589860 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" event={"ID":"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d","Type":"ContainerStarted","Data":"7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d"} Aug 04 16:09:20.590376 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:20.590006 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:20.614194 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:20.614142 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" podStartSLOduration=1.8676708400000002 podStartE2EDuration="4.614127011s" totalImagesPullingTime="2.746456171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:09:20.613630644 +0000 UTC m=+789.936054124" watchObservedRunningTime="2026-08-04 16:09:20.614127011 +0000 UTC m=+789.936550490" Aug 04 16:09:30.910862 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:30.910831 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-974d9c7f7-jtmrf"] Aug 04 16:09:30.911378 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:30.911015 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" podUID="7f8bc7dd-b0e1-44ce-9afd-a406d256c93d" containerName="manager" containerID="cri-o://7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d" gracePeriod=30 Aug 04 16:09:30.912644 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:30.912622 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:31.162554 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.162492 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:31.317573 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.317527 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-webhook-cert\") pod \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\" (UID: \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\") " Aug 04 16:09:31.317807 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.317625 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsk25\" (UniqueName: \"kubernetes.io/projected/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-kube-api-access-fsk25\") pod \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\" (UID: \"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d\") " Aug 04 16:09:31.319871 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.319846 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-kube-api-access-fsk25" (OuterVolumeSpecName: "kube-api-access-fsk25") pod "7f8bc7dd-b0e1-44ce-9afd-a406d256c93d" (UID: "7f8bc7dd-b0e1-44ce-9afd-a406d256c93d"). InnerVolumeSpecName "kube-api-access-fsk25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:09:31.319981 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.319967 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7f8bc7dd-b0e1-44ce-9afd-a406d256c93d" (UID: "7f8bc7dd-b0e1-44ce-9afd-a406d256c93d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:09:31.419443 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.419349 2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-webhook-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:09:31.419443 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.419384 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsk25\" (UniqueName: \"kubernetes.io/projected/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d-kube-api-access-fsk25\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:09:31.642420 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.642385 2556 generic.go:356] "Generic (PLEG): container finished" podID="7f8bc7dd-b0e1-44ce-9afd-a406d256c93d" containerID="7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d" exitCode=0 Aug 04 16:09:31.642588 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.642430 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" Aug 04 16:09:31.642588 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.642455 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" event={"ID":"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d","Type":"ContainerDied","Data":"7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d"} Aug 04 16:09:31.642588 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.642497 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-974d9c7f7-jtmrf" event={"ID":"7f8bc7dd-b0e1-44ce-9afd-a406d256c93d","Type":"ContainerDied","Data":"41ae88f0d5babd5b13f1ec9256b3f0ee814c1ace2c7c414441b9d896c487e7a5"} Aug 04 16:09:31.642588 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.642520 2556 scope.go:117] "RemoveContainer" containerID="7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d" Aug 04 16:09:31.651402 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.651386 2556 scope.go:117] "RemoveContainer" containerID="7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d" Aug 04 16:09:31.651648 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:09:31.651627 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d\": container with ID starting with 7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d not found: ID does not exist" containerID="7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d" Aug 04 16:09:31.651699 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.651655 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d"} err="failed to get container status \"7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d\": rpc error: code = NotFound desc = could not find container \"7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d\": container with ID starting with 7cfbc1452c29762f8d658c2c4e7480a9fbff3e65ce73a317b87d6236e821f55d not found: ID does not exist" Aug 04 16:09:31.665509 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.665480 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-974d9c7f7-jtmrf"] Aug 04 16:09:31.671826 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:31.671773 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-974d9c7f7-jtmrf"] Aug 04 16:09:33.273458 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:33.273379 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f8bc7dd-b0e1-44ce-9afd-a406d256c93d" path="/var/lib/kubelet/pods/7f8bc7dd-b0e1-44ce-9afd-a406d256c93d/volumes" Aug 04 16:09:55.442352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.442311 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-6vftr"] Aug 04 16:09:55.442744 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.442711 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8bc7dd-b0e1-44ce-9afd-a406d256c93d" containerName="manager" Aug 04 16:09:55.442811 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.442745 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8bc7dd-b0e1-44ce-9afd-a406d256c93d" containerName="manager" Aug 04 16:09:55.442937 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.442918 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8bc7dd-b0e1-44ce-9afd-a406d256c93d" containerName="manager" Aug 04 16:09:55.451147 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.451120 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:09:55.454198 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.454170 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Aug 04 16:09:55.454367 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.454171 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-r2mvf\"" Aug 04 16:09:55.464820 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.464789 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-6vftr"] Aug 04 16:09:55.535841 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.535810 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2hjp\" (UniqueName: \"kubernetes.io/projected/494ec520-130e-4ca2-a9ca-4f11a19f69eb-kube-api-access-t2hjp\") pod \"payload-processing-5cc6cd444b-6vftr\" (UID: \"494ec520-130e-4ca2-a9ca-4f11a19f69eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:09:55.536022 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.535878 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/494ec520-130e-4ca2-a9ca-4f11a19f69eb-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-6vftr\" (UID: \"494ec520-130e-4ca2-a9ca-4f11a19f69eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:09:55.637478 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.637432 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/494ec520-130e-4ca2-a9ca-4f11a19f69eb-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-6vftr\" (UID: \"494ec520-130e-4ca2-a9ca-4f11a19f69eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:09:55.637640 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.637588 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2hjp\" (UniqueName: \"kubernetes.io/projected/494ec520-130e-4ca2-a9ca-4f11a19f69eb-kube-api-access-t2hjp\") pod \"payload-processing-5cc6cd444b-6vftr\" (UID: \"494ec520-130e-4ca2-a9ca-4f11a19f69eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:09:55.638254 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.638233 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/494ec520-130e-4ca2-a9ca-4f11a19f69eb-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-6vftr\" (UID: \"494ec520-130e-4ca2-a9ca-4f11a19f69eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:09:55.653031 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.652985 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2hjp\" (UniqueName: \"kubernetes.io/projected/494ec520-130e-4ca2-a9ca-4f11a19f69eb-kube-api-access-t2hjp\") pod \"payload-processing-5cc6cd444b-6vftr\" (UID: \"494ec520-130e-4ca2-a9ca-4f11a19f69eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:09:55.763146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.763111 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:09:55.913046 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:55.913016 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-6vftr"] Aug 04 16:09:55.914847 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:09:55.914816 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494ec520_130e_4ca2_a9ca_4f11a19f69eb.slice/crio-50c548f3ac9e52aec3067689047b17e1cc3fd3f8123bba62921698fb489cb752 WatchSource:0}: Error finding container 50c548f3ac9e52aec3067689047b17e1cc3fd3f8123bba62921698fb489cb752: Status 404 returned error can't find the container with id 50c548f3ac9e52aec3067689047b17e1cc3fd3f8123bba62921698fb489cb752 Aug 04 16:09:56.762974 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:56.762940 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" event={"ID":"494ec520-130e-4ca2-a9ca-4f11a19f69eb","Type":"ContainerStarted","Data":"50c548f3ac9e52aec3067689047b17e1cc3fd3f8123bba62921698fb489cb752"} Aug 04 16:09:59.782240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:59.782181 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" event={"ID":"494ec520-130e-4ca2-a9ca-4f11a19f69eb","Type":"ContainerStarted","Data":"ec4e939fef1007ba98e9ec9314c2726cbb9522cbe22d112c703058287eeedbae"} Aug 04 16:09:59.783839 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:59.783807 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:09:59.805868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:09:59.805815 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" podStartSLOduration=1.497703502 podStartE2EDuration="4.805800508s" totalImagesPullingTime="3.308097006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:09:59.802915336 +0000 UTC m=+829.125338826" watchObservedRunningTime="2026-08-04 16:09:59.805800508 +0000 UTC m=+829.128224021" Aug 04 16:10:04.809081 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:04.809046 2556 generic.go:356] "Generic (PLEG): container finished" podID="9f3fb966-8f4b-4709-bbbf-1f45529c4eaf" containerID="6cf91f21ba9ee01d2fc30874017c9115a37facbfc36ed0940edaa33b73ae8934" exitCode=0 Aug 04 16:10:04.809600 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:04.809125 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" event={"ID":"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf","Type":"ContainerDied","Data":"6cf91f21ba9ee01d2fc30874017c9115a37facbfc36ed0940edaa33b73ae8934"} Aug 04 16:10:04.809666 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:04.809647 2556 scope.go:117] "RemoveContainer" containerID="6cf91f21ba9ee01d2fc30874017c9115a37facbfc36ed0940edaa33b73ae8934" Aug 04 16:10:04.810678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:04.810648 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:10:04.810804 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:04.810755 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:10:04.810846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:04.810801 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:10:05.816498 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:05.816452 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" event={"ID":"9f3fb966-8f4b-4709-bbbf-1f45529c4eaf","Type":"ContainerStarted","Data":"078cdbaaec27ab12d2c33a6fa1358ee1fc873976c8513d6ba7cf11bbe19ecd35"} Aug 04 16:10:06.239815 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:06.239777 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:10:06.241178 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:06.241145 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7gqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Aug 04 16:10:06.241427 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:06.241394 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" podUID="9f3fb966-8f4b-4709-bbbf-1f45529c4eaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Aug 04 16:10:07.240704 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:07.240665 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7gqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Aug 04 16:10:07.241137 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:07.240750 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" podUID="9f3fb966-8f4b-4709-bbbf-1f45529c4eaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Aug 04 16:10:08.240367 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:08.240329 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7gqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Aug 04 16:10:08.240693 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:08.240417 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" podUID="9f3fb966-8f4b-4709-bbbf-1f45529c4eaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Aug 04 16:10:09.240925 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:09.240883 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7gqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Aug 04 16:10:09.241345 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:09.240942 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" podUID="9f3fb966-8f4b-4709-bbbf-1f45529c4eaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Aug 04 16:10:09.345164 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:09.345116 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:10:10.240683 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:10.240647 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7gqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Aug 04 16:10:10.240885 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:10.240703 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" podUID="9f3fb966-8f4b-4709-bbbf-1f45529c4eaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Aug 04 16:10:11.241140 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:11.241089 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7gqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Aug 04 16:10:11.241526 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:11.241167 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" podUID="9f3fb966-8f4b-4709-bbbf-1f45529c4eaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Aug 04 16:10:11.796908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:11.796873 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-6vftr" Aug 04 16:10:12.243908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:12.243877 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:10:12.244817 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:12.244789 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7gqdr" Aug 04 16:10:13.493589 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.493556 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v"] Aug 04 16:10:13.498312 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.498295 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.501118 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.501096 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 16:10:13.501118 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.501111 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 16:10:13.502483 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.502467 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Aug 04 16:10:13.502571 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.502493 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zcdht\"" Aug 04 16:10:13.506250 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.506226 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v"] Aug 04 16:10:13.620464 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.620420 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjdt4\" (UniqueName: \"kubernetes.io/projected/8857eb2c-576d-42d7-87ec-9bfdaa387011-kube-api-access-xjdt4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.620464 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.620470 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.620681 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.620509 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.620681 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.620524 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.620681 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.620538 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8857eb2c-576d-42d7-87ec-9bfdaa387011-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.620838 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.620687 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.721762 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.721702 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.721937 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.721785 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjdt4\" (UniqueName: \"kubernetes.io/projected/8857eb2c-576d-42d7-87ec-9bfdaa387011-kube-api-access-xjdt4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.721937 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.721813 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.721937 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.721832 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.721937 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.721857 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.722138 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.721999 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8857eb2c-576d-42d7-87ec-9bfdaa387011-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.722138 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.722116 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.722213 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.722166 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.722213 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.722195 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.724075 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.724056 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8857eb2c-576d-42d7-87ec-9bfdaa387011-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.724317 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.724296 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8857eb2c-576d-42d7-87ec-9bfdaa387011-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.730436 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.730412 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjdt4\" (UniqueName: \"kubernetes.io/projected/8857eb2c-576d-42d7-87ec-9bfdaa387011-kube-api-access-xjdt4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v\" (UID: \"8857eb2c-576d-42d7-87ec-9bfdaa387011\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.809785 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.809676 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:13.982283 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:13.982256 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v"] Aug 04 16:10:13.983854 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:10:13.983823 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8857eb2c_576d_42d7_87ec_9bfdaa387011.slice/crio-940da7cdcdc924974afbfc2d2a440633fbe59af1ce835b2773b6bf95ea09b491 WatchSource:0}: Error finding container 940da7cdcdc924974afbfc2d2a440633fbe59af1ce835b2773b6bf95ea09b491: Status 404 returned error can't find the container with id 940da7cdcdc924974afbfc2d2a440633fbe59af1ce835b2773b6bf95ea09b491 Aug 04 16:10:14.869482 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:14.869440 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" event={"ID":"8857eb2c-576d-42d7-87ec-9bfdaa387011","Type":"ContainerStarted","Data":"940da7cdcdc924974afbfc2d2a440633fbe59af1ce835b2773b6bf95ea09b491"} Aug 04 16:10:15.876789 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:15.876747 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" event={"ID":"8857eb2c-576d-42d7-87ec-9bfdaa387011","Type":"ContainerStarted","Data":"09b64ec55f762175454dc9f45d434fe2d56bd6947641aa557d76d645ee1ae8a0"} Aug 04 16:10:15.899277 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:15.899226 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" podStartSLOduration=1.605524488 podStartE2EDuration="2.899208008s" totalImagesPullingTime="1.29368352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:10:15.897153917 +0000 UTC m=+845.219577410" watchObservedRunningTime="2026-08-04 16:10:15.899208008 +0000 UTC m=+845.221631512" Aug 04 16:10:23.472519 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.472487 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv"] Aug 04 16:10:23.476261 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.476239 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.478995 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.478963 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Aug 04 16:10:23.489509 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.489487 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv"] Aug 04 16:10:23.614123 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.614078 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ab4cf74-591e-412d-9416-e9dcee87f394-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.614309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.614184 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.614309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.614229 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.614309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.614275 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.614309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.614291 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.614309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.614309 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh28k\" (UniqueName: \"kubernetes.io/projected/6ab4cf74-591e-412d-9416-e9dcee87f394-kube-api-access-wh28k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.715643 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.715601 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.715832 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.715654 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.715832 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.715697 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.715832 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.715714 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.715832 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.715767 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh28k\" (UniqueName: \"kubernetes.io/projected/6ab4cf74-591e-412d-9416-e9dcee87f394-kube-api-access-wh28k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.715832 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.715813 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ab4cf74-591e-412d-9416-e9dcee87f394-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.716152 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.716131 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.716230 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.716210 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.716301 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.716281 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.717959 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.717936 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ab4cf74-591e-412d-9416-e9dcee87f394-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.718239 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.718221 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ab4cf74-591e-412d-9416-e9dcee87f394-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.725049 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.724994 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh28k\" (UniqueName: \"kubernetes.io/projected/6ab4cf74-591e-412d-9416-e9dcee87f394-kube-api-access-wh28k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-48cgv\" (UID: \"6ab4cf74-591e-412d-9416-e9dcee87f394\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.786735 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.786684 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:23.810092 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.810059 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:23.810202 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.810138 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:23.822735 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.822692 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:23.929451 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.929423 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v" Aug 04 16:10:23.945307 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:23.945281 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv"] Aug 04 16:10:23.947295 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:10:23.947270 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab4cf74_591e_412d_9416_e9dcee87f394.slice/crio-148744e943b804592d9bf31bc19db99a46bb577f9e135c1b7ba02c641eff6d62 WatchSource:0}: Error finding container 148744e943b804592d9bf31bc19db99a46bb577f9e135c1b7ba02c641eff6d62: Status 404 returned error can't find the container with id 148744e943b804592d9bf31bc19db99a46bb577f9e135c1b7ba02c641eff6d62 Aug 04 16:10:24.923368 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:24.923331 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" event={"ID":"6ab4cf74-591e-412d-9416-e9dcee87f394","Type":"ContainerStarted","Data":"0181d9cbe550a68d16b09f821aa2dcc3ef9ec0e6094b226b114d9c87d23a6d12"} Aug 04 16:10:24.923788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:24.923422 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" event={"ID":"6ab4cf74-591e-412d-9416-e9dcee87f394","Type":"ContainerStarted","Data":"148744e943b804592d9bf31bc19db99a46bb577f9e135c1b7ba02c641eff6d62"} Aug 04 16:10:24.946713 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:24.946651 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" podStartSLOduration=1.60576378 podStartE2EDuration="1.946636072s" totalImagesPullingTime="340.872292ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:10:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:10:24.944438739 +0000 UTC m=+854.266862220" watchObservedRunningTime="2026-08-04 16:10:24.946636072 +0000 UTC m=+854.269059553" Aug 04 16:10:33.786900 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:33.786853 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:33.787320 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:33.786928 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:33.799510 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:33.799481 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:10:33.976864 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:10:33.976829 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-48cgv" Aug 04 16:12:07.613285 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:07.613228 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75b9569c58-zm8r7"] Aug 04 16:12:07.613876 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:07.613504 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75b9569c58-zm8r7" podUID="d70f1e47-c0c4-4460-a485-0f7eac5a4114" containerName="authorino" containerID="cri-o://692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a" gracePeriod=30 Aug 04 16:12:07.869873 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:07.869803 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:12:07.979581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:07.979544 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d70f1e47-c0c4-4460-a485-0f7eac5a4114-tls-cert\") pod \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\" (UID: \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\") " Aug 04 16:12:07.979826 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:07.979597 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq2dr\" (UniqueName: \"kubernetes.io/projected/d70f1e47-c0c4-4460-a485-0f7eac5a4114-kube-api-access-wq2dr\") pod \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\" (UID: \"d70f1e47-c0c4-4460-a485-0f7eac5a4114\") " Aug 04 16:12:07.981768 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:07.981737 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d70f1e47-c0c4-4460-a485-0f7eac5a4114-kube-api-access-wq2dr" (OuterVolumeSpecName: "kube-api-access-wq2dr") pod "d70f1e47-c0c4-4460-a485-0f7eac5a4114" (UID: "d70f1e47-c0c4-4460-a485-0f7eac5a4114"). InnerVolumeSpecName "kube-api-access-wq2dr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:12:07.989996 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:07.989966 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d70f1e47-c0c4-4460-a485-0f7eac5a4114-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d70f1e47-c0c4-4460-a485-0f7eac5a4114" (UID: "d70f1e47-c0c4-4460-a485-0f7eac5a4114"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:12:08.081161 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.081121 2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d70f1e47-c0c4-4460-a485-0f7eac5a4114-tls-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:12:08.081161 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.081153 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq2dr\" (UniqueName: \"kubernetes.io/projected/d70f1e47-c0c4-4460-a485-0f7eac5a4114-kube-api-access-wq2dr\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:12:08.428444 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.428408 2556 generic.go:356] "Generic (PLEG): container finished" podID="d70f1e47-c0c4-4460-a485-0f7eac5a4114" containerID="692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a" exitCode=0 Aug 04 16:12:08.428444 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.428446 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b9569c58-zm8r7" Aug 04 16:12:08.428679 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.428478 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b9569c58-zm8r7" event={"ID":"d70f1e47-c0c4-4460-a485-0f7eac5a4114","Type":"ContainerDied","Data":"692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a"} Aug 04 16:12:08.428679 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.428511 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b9569c58-zm8r7" event={"ID":"d70f1e47-c0c4-4460-a485-0f7eac5a4114","Type":"ContainerDied","Data":"afc164087cb8ea91ead80fdd50c44d4ff847fc32c9ce4bb486ebf79836065728"} Aug 04 16:12:08.428679 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.428527 2556 scope.go:117] "RemoveContainer" containerID="692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a" Aug 04 16:12:08.438214 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.438196 2556 scope.go:117] "RemoveContainer" containerID="692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a" Aug 04 16:12:08.438517 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:12:08.438494 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a\": container with ID starting with 692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a not found: ID does not exist" containerID="692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a" Aug 04 16:12:08.438567 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.438526 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a"} err="failed to get container status \"692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a\": rpc error: code = NotFound desc = could not find container \"692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a\": container with ID starting with 692c8e0679df34f65841630f96576144e36f6c048a31cb5080f8699e6dfb868a not found: ID does not exist" Aug 04 16:12:08.454291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.454257 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75b9569c58-zm8r7"] Aug 04 16:12:08.458399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:08.458368 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75b9569c58-zm8r7"] Aug 04 16:12:09.273059 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:12:09.273023 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d70f1e47-c0c4-4460-a485-0f7eac5a4114" path="/var/lib/kubelet/pods/d70f1e47-c0c4-4460-a485-0f7eac5a4114/volumes" Aug 04 16:13:58.680112 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.680061 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d474f94c9-kw28d"] Aug 04 16:13:58.680581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.680465 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d70f1e47-c0c4-4460-a485-0f7eac5a4114" containerName="authorino" Aug 04 16:13:58.680581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.680476 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d70f1e47-c0c4-4460-a485-0f7eac5a4114" containerName="authorino" Aug 04 16:13:58.680581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.680550 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d70f1e47-c0c4-4460-a485-0f7eac5a4114" containerName="authorino" Aug 04 16:13:58.683530 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.683513 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:13:58.687343 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.687317 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-42s8l\"" Aug 04 16:13:58.687520 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.687367 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 16:13:58.687520 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.687317 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 16:13:58.691278 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.691251 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20aa7902-1cd0-4e36-a96f-b1957070cbc9-webhook-cert\") pod \"maas-controller-d474f94c9-kw28d\" (UID: \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\") " pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:13:58.691278 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.691308 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw4n8\" (UniqueName: \"kubernetes.io/projected/20aa7902-1cd0-4e36-a96f-b1957070cbc9-kube-api-access-nw4n8\") pod \"maas-controller-d474f94c9-kw28d\" (UID: \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\") " pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:13:58.693873 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.693846 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d474f94c9-kw28d"] Aug 04 16:13:58.792191 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.792156 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw4n8\" (UniqueName: \"kubernetes.io/projected/20aa7902-1cd0-4e36-a96f-b1957070cbc9-kube-api-access-nw4n8\") pod \"maas-controller-d474f94c9-kw28d\" (UID: \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\") " pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:13:58.792400 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.792277 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20aa7902-1cd0-4e36-a96f-b1957070cbc9-webhook-cert\") pod \"maas-controller-d474f94c9-kw28d\" (UID: \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\") " pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:13:58.794757 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.794715 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20aa7902-1cd0-4e36-a96f-b1957070cbc9-webhook-cert\") pod \"maas-controller-d474f94c9-kw28d\" (UID: \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\") " pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:13:58.802846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.802814 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw4n8\" (UniqueName: \"kubernetes.io/projected/20aa7902-1cd0-4e36-a96f-b1957070cbc9-kube-api-access-nw4n8\") pod \"maas-controller-d474f94c9-kw28d\" (UID: \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\") " pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:13:58.995563 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:58.995530 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:13:59.133250 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:59.133176 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d474f94c9-kw28d"] Aug 04 16:13:59.135563 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:13:59.135535 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20aa7902_1cd0_4e36_a96f_b1957070cbc9.slice/crio-2d8b4319160a870f9a323ec5df64ee6e9eee06ef018b629b6abfcc65da2a1cfd WatchSource:0}: Error finding container 2d8b4319160a870f9a323ec5df64ee6e9eee06ef018b629b6abfcc65da2a1cfd: Status 404 returned error can't find the container with id 2d8b4319160a870f9a323ec5df64ee6e9eee06ef018b629b6abfcc65da2a1cfd Aug 04 16:13:59.137001 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:59.136980 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 16:13:59.955189 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:59.955088 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d474f94c9-kw28d" event={"ID":"20aa7902-1cd0-4e36-a96f-b1957070cbc9","Type":"ContainerStarted","Data":"0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a"} Aug 04 16:13:59.955189 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:59.955170 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d474f94c9-kw28d" event={"ID":"20aa7902-1cd0-4e36-a96f-b1957070cbc9","Type":"ContainerStarted","Data":"2d8b4319160a870f9a323ec5df64ee6e9eee06ef018b629b6abfcc65da2a1cfd"} Aug 04 16:13:59.955588 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:59.955199 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:13:59.975843 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:13:59.975794 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d474f94c9-kw28d" podStartSLOduration=1.5515376440000002 podStartE2EDuration="1.975778854s" totalImagesPullingTime="424.24121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:13:59.973869228 +0000 UTC m=+1069.296292719" watchObservedRunningTime="2026-08-04 16:13:59.975778854 +0000 UTC m=+1069.298202341" Aug 04 16:14:10.963203 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:10.963123 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:14:15.692430 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:15.692389 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d474f94c9-kw28d"] Aug 04 16:14:15.692903 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:15.692583 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d474f94c9-kw28d" podUID="20aa7902-1cd0-4e36-a96f-b1957070cbc9" containerName="manager" containerID="cri-o://0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a" gracePeriod=30 Aug 04 16:14:15.944048 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:15.943990 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:14:16.037167 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.037134 2556 generic.go:356] "Generic (PLEG): container finished" podID="20aa7902-1cd0-4e36-a96f-b1957070cbc9" containerID="0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a" exitCode=0 Aug 04 16:14:16.037167 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.037172 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d474f94c9-kw28d" event={"ID":"20aa7902-1cd0-4e36-a96f-b1957070cbc9","Type":"ContainerDied","Data":"0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a"} Aug 04 16:14:16.037378 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.037175 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d474f94c9-kw28d" Aug 04 16:14:16.037378 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.037211 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d474f94c9-kw28d" event={"ID":"20aa7902-1cd0-4e36-a96f-b1957070cbc9","Type":"ContainerDied","Data":"2d8b4319160a870f9a323ec5df64ee6e9eee06ef018b629b6abfcc65da2a1cfd"} Aug 04 16:14:16.037378 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.037235 2556 scope.go:117] "RemoveContainer" containerID="0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a" Aug 04 16:14:16.045575 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.045551 2556 scope.go:117] "RemoveContainer" containerID="0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a" Aug 04 16:14:16.045960 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:14:16.045938 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a\": container with ID starting with 0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a not found: ID does not exist" containerID="0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a" Aug 04 16:14:16.046018 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.045969 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a"} err="failed to get container status \"0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a\": rpc error: code = NotFound desc = could not find container \"0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a\": container with ID starting with 0f4ce5edcad49ecc4615f3c21fe7b91891d1060eb355ee387698fb8606432a8a not found: ID does not exist" Aug 04 16:14:16.059291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.059264 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20aa7902-1cd0-4e36-a96f-b1957070cbc9-webhook-cert\") pod \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\" (UID: \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\") " Aug 04 16:14:16.059417 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.059370 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw4n8\" (UniqueName: \"kubernetes.io/projected/20aa7902-1cd0-4e36-a96f-b1957070cbc9-kube-api-access-nw4n8\") pod \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\" (UID: \"20aa7902-1cd0-4e36-a96f-b1957070cbc9\") " Aug 04 16:14:16.061216 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.061189 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20aa7902-1cd0-4e36-a96f-b1957070cbc9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "20aa7902-1cd0-4e36-a96f-b1957070cbc9" (UID: "20aa7902-1cd0-4e36-a96f-b1957070cbc9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:14:16.061315 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.061268 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20aa7902-1cd0-4e36-a96f-b1957070cbc9-kube-api-access-nw4n8" (OuterVolumeSpecName: "kube-api-access-nw4n8") pod "20aa7902-1cd0-4e36-a96f-b1957070cbc9" (UID: "20aa7902-1cd0-4e36-a96f-b1957070cbc9"). InnerVolumeSpecName "kube-api-access-nw4n8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:14:16.160377 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.160329 2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20aa7902-1cd0-4e36-a96f-b1957070cbc9-webhook-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:14:16.160377 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.160373 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw4n8\" (UniqueName: \"kubernetes.io/projected/20aa7902-1cd0-4e36-a96f-b1957070cbc9-kube-api-access-nw4n8\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:14:16.359684 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.359652 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d474f94c9-kw28d"] Aug 04 16:14:16.364526 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:16.364500 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d474f94c9-kw28d"] Aug 04 16:14:17.273354 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.273318 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20aa7902-1cd0-4e36-a96f-b1957070cbc9" path="/var/lib/kubelet/pods/20aa7902-1cd0-4e36-a96f-b1957070cbc9/volumes" Aug 04 16:14:17.300450 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.300412 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d474f94c9-frkfm"] Aug 04 16:14:17.300865 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.300839 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20aa7902-1cd0-4e36-a96f-b1957070cbc9" containerName="manager" Aug 04 16:14:17.300865 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.300866 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="20aa7902-1cd0-4e36-a96f-b1957070cbc9" containerName="manager" Aug 04 16:14:17.300971 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.300928 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="20aa7902-1cd0-4e36-a96f-b1957070cbc9" containerName="manager" Aug 04 16:14:17.305392 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.305369 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:14:17.308245 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.308222 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-42s8l\"" Aug 04 16:14:17.309291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.309273 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 16:14:17.309389 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.309291 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 16:14:17.315129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.315103 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d474f94c9-frkfm"] Aug 04 16:14:17.371289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.371253 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4792e00-f3e3-4ddf-9bf6-6e39924d9129-webhook-cert\") pod \"maas-controller-d474f94c9-frkfm\" (UID: \"d4792e00-f3e3-4ddf-9bf6-6e39924d9129\") " pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:14:17.371481 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.371316 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx7d5\" (UniqueName: \"kubernetes.io/projected/d4792e00-f3e3-4ddf-9bf6-6e39924d9129-kube-api-access-lx7d5\") pod \"maas-controller-d474f94c9-frkfm\" (UID: \"d4792e00-f3e3-4ddf-9bf6-6e39924d9129\") " pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:14:17.472405 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.472367 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx7d5\" (UniqueName: \"kubernetes.io/projected/d4792e00-f3e3-4ddf-9bf6-6e39924d9129-kube-api-access-lx7d5\") pod \"maas-controller-d474f94c9-frkfm\" (UID: \"d4792e00-f3e3-4ddf-9bf6-6e39924d9129\") " pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:14:17.472590 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.472478 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4792e00-f3e3-4ddf-9bf6-6e39924d9129-webhook-cert\") pod \"maas-controller-d474f94c9-frkfm\" (UID: \"d4792e00-f3e3-4ddf-9bf6-6e39924d9129\") " pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:14:17.475185 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.475157 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4792e00-f3e3-4ddf-9bf6-6e39924d9129-webhook-cert\") pod \"maas-controller-d474f94c9-frkfm\" (UID: \"d4792e00-f3e3-4ddf-9bf6-6e39924d9129\") " pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:14:17.482395 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.482367 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx7d5\" (UniqueName: \"kubernetes.io/projected/d4792e00-f3e3-4ddf-9bf6-6e39924d9129-kube-api-access-lx7d5\") pod \"maas-controller-d474f94c9-frkfm\" (UID: \"d4792e00-f3e3-4ddf-9bf6-6e39924d9129\") " pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:14:17.617858 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.617769 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:14:17.753745 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:17.753696 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d474f94c9-frkfm"] Aug 04 16:14:17.757175 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:14:17.757148 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4792e00_f3e3_4ddf_9bf6_6e39924d9129.slice/crio-2493d89c23326126c308ffdd6e90c6a404aea1ea9dab509c5743772ec78d5836 WatchSource:0}: Error finding container 2493d89c23326126c308ffdd6e90c6a404aea1ea9dab509c5743772ec78d5836: Status 404 returned error can't find the container with id 2493d89c23326126c308ffdd6e90c6a404aea1ea9dab509c5743772ec78d5836 Aug 04 16:14:18.048943 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:18.048904 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d474f94c9-frkfm" event={"ID":"d4792e00-f3e3-4ddf-9bf6-6e39924d9129","Type":"ContainerStarted","Data":"2493d89c23326126c308ffdd6e90c6a404aea1ea9dab509c5743772ec78d5836"} Aug 04 16:14:19.055021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:19.054981 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d474f94c9-frkfm" event={"ID":"d4792e00-f3e3-4ddf-9bf6-6e39924d9129","Type":"ContainerStarted","Data":"2d80a68363bf3662e3d50646d04a9c1f1de97a44d17182f5273b7b8f72ff5ce9"} Aug 04 16:14:19.055519 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:19.055043 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:14:19.077781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:19.077716 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d474f94c9-frkfm" podStartSLOduration=1.723906849 podStartE2EDuration="2.077699179s" totalImagesPullingTime="353.79233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:14:19.075545252 +0000 UTC m=+1088.397968757" watchObservedRunningTime="2026-08-04 16:14:19.077699179 +0000 UTC m=+1088.400122659" Aug 04 16:14:30.062111 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:14:30.062073 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d474f94c9-frkfm" Aug 04 16:15:00.149204 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:00.149167 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd"] Aug 04 16:15:00.152892 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:00.152865 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd" Aug 04 16:15:00.155637 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:00.155607 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-psz5k\"" Aug 04 16:15:00.162132 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:00.161607 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd"] Aug 04 16:15:00.165112 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:00.165086 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd" Aug 04 16:15:00.309294 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:00.309264 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd"] Aug 04 16:15:00.310854 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:15:00.310823 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode663aa28_702d_4637_b668_12821de6232f.slice/crio-a441fccdf3a09b058284f0c9856d25817ce804c07067a7b0f8336f3bfccdd85f WatchSource:0}: Error finding container a441fccdf3a09b058284f0c9856d25817ce804c07067a7b0f8336f3bfccdd85f: Status 404 returned error can't find the container with id a441fccdf3a09b058284f0c9856d25817ce804c07067a7b0f8336f3bfccdd85f Aug 04 16:15:01.264543 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:01.264505 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd" event={"ID":"e663aa28-702d-4637-b668-12821de6232f","Type":"ContainerStarted","Data":"a441fccdf3a09b058284f0c9856d25817ce804c07067a7b0f8336f3bfccdd85f"} Aug 04 16:15:05.286252 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:05.286209 2556 generic.go:356] "Generic (PLEG): container finished" podID="e663aa28-702d-4637-b668-12821de6232f" containerID="e0a5b5124a8c03303c7c9304e2875e02608e5e2aa0da06c1d92bb3ffb37c4fe4" exitCode=0 Aug 04 16:15:05.286673 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:05.286258 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd" event={"ID":"e663aa28-702d-4637-b668-12821de6232f","Type":"ContainerDied","Data":"e0a5b5124a8c03303c7c9304e2875e02608e5e2aa0da06c1d92bb3ffb37c4fe4"} Aug 04 16:15:06.428007 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:06.427981 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd" Aug 04 16:15:07.297500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:07.297468 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd" Aug 04 16:15:07.297658 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:07.297469 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd" event={"ID":"e663aa28-702d-4637-b668-12821de6232f","Type":"ContainerDied","Data":"a441fccdf3a09b058284f0c9856d25817ce804c07067a7b0f8336f3bfccdd85f"} Aug 04 16:15:07.297658 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:15:07.297599 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a441fccdf3a09b058284f0c9856d25817ce804c07067a7b0f8336f3bfccdd85f" Aug 04 16:25:11.009797 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.009764 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx"] Aug 04 16:25:11.010405 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.010342 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e663aa28-702d-4637-b668-12821de6232f" containerName="cleanup" Aug 04 16:25:11.010405 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.010364 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e663aa28-702d-4637-b668-12821de6232f" containerName="cleanup" Aug 04 16:25:11.010532 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.010493 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e663aa28-702d-4637-b668-12821de6232f" containerName="cleanup" Aug 04 16:25:11.013790 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.013768 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.016880 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.016851 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-34eb1536-kserve-self-signed-certs\"" Aug 04 16:25:11.036825 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.036789 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx"] Aug 04 16:25:11.116575 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.116540 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-home\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.116575 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.116575 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-tmp-dir\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.116842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.116603 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-model-cache\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.116842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.116621 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-dshm\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.116842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.116682 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e58ac783-4909-4a44-8b2f-56353ac5202f-tls-certs\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.116842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.116802 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72h6d\" (UniqueName: \"kubernetes.io/projected/e58ac783-4909-4a44-8b2f-56353ac5202f-kube-api-access-72h6d\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.218166 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.218127 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e58ac783-4909-4a44-8b2f-56353ac5202f-tls-certs\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.218349 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.218193 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72h6d\" (UniqueName: \"kubernetes.io/projected/e58ac783-4909-4a44-8b2f-56353ac5202f-kube-api-access-72h6d\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.218349 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.218231 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-home\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.218349 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.218255 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-tmp-dir\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.218349 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.218295 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-model-cache\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.218548 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.218432 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-dshm\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.218644 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.218628 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-model-cache\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.218720 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.218696 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-tmp-dir\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.218720 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.218715 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-home\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.220570 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.220550 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-dshm\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.221163 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.221146 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-34eb1536-kserve-self-signed-certs\"" Aug 04 16:25:11.227131 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.227105 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72h6d\" (UniqueName: \"kubernetes.io/projected/e58ac783-4909-4a44-8b2f-56353ac5202f-kube-api-access-72h6d\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.230983 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.230958 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e58ac783-4909-4a44-8b2f-56353ac5202f-tls-certs\") pod \"e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.325622 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.325527 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:11.465109 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:11.465075 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx"] Aug 04 16:25:11.468450 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:25:11.468424 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode58ac783_4909_4a44_8b2f_56353ac5202f.slice/crio-6e5e28b994fa34c037aaf2afbf04544dc4df04ab2a95e5f9c0928770cd1d9597 WatchSource:0}: Error finding container 6e5e28b994fa34c037aaf2afbf04544dc4df04ab2a95e5f9c0928770cd1d9597: Status 404 returned error can't find the container with id 6e5e28b994fa34c037aaf2afbf04544dc4df04ab2a95e5f9c0928770cd1d9597 Aug 04 16:25:12.323887 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:12.323844 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" event={"ID":"e58ac783-4909-4a44-8b2f-56353ac5202f","Type":"ContainerStarted","Data":"ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670"} Aug 04 16:25:12.324378 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:12.323900 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" event={"ID":"e58ac783-4909-4a44-8b2f-56353ac5202f","Type":"ContainerStarted","Data":"6e5e28b994fa34c037aaf2afbf04544dc4df04ab2a95e5f9c0928770cd1d9597"} Aug 04 16:25:12.345038 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:12.344982 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" podStartSLOduration=2.34496635 podStartE2EDuration="2.34496635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:25:12.342935892 +0000 UTC m=+1741.665359373" watchObservedRunningTime="2026-08-04 16:25:12.34496635 +0000 UTC m=+1741.667389829" Aug 04 16:25:21.326163 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:21.326117 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:21.326606 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:21.326182 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:21.338933 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:21.338896 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:21.382978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:21.382936 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:25:36.881215 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:36.881179 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7"] Aug 04 16:25:36.885531 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:36.885507 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:36.888271 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:36.888249 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-34eb1536-kserve-self-signed-certs\"" Aug 04 16:25:36.898653 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:36.898598 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7"] Aug 04 16:25:37.062274 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.062225 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-model-cache\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.062471 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.062294 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-tmp-dir\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.062471 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.062341 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-dshm\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.062471 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.062364 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45h8g\" (UniqueName: \"kubernetes.io/projected/eeca3752-e828-48ff-939c-ec5e1d83532c-kube-api-access-45h8g\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.062471 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.062386 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeca3752-e828-48ff-939c-ec5e1d83532c-tls-certs\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.062471 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.062425 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-home\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.164022 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.163907 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-model-cache\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.164022 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.164017 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-tmp-dir\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.164249 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.164074 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-dshm\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.164249 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.164202 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45h8g\" (UniqueName: \"kubernetes.io/projected/eeca3752-e828-48ff-939c-ec5e1d83532c-kube-api-access-45h8g\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.164351 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.164247 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeca3752-e828-48ff-939c-ec5e1d83532c-tls-certs\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.164351 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.164299 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-home\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.164458 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.164395 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-model-cache\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.164458 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.164432 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-tmp-dir\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.164665 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.164639 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-home\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.166383 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.166358 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-dshm\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.166640 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.166623 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeca3752-e828-48ff-939c-ec5e1d83532c-tls-certs\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.179156 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.179130 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45h8g\" (UniqueName: \"kubernetes.io/projected/eeca3752-e828-48ff-939c-ec5e1d83532c-kube-api-access-45h8g\") pod \"e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.196033 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.195999 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:37.346581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.346547 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7"] Aug 04 16:25:37.348631 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:25:37.348603 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeeca3752_e828_48ff_939c_ec5e1d83532c.slice/crio-2e51c65521097a9bc179aaf1d4ca988c7ee3cda41591f2c20699c06826c89fe3 WatchSource:0}: Error finding container 2e51c65521097a9bc179aaf1d4ca988c7ee3cda41591f2c20699c06826c89fe3: Status 404 returned error can't find the container with id 2e51c65521097a9bc179aaf1d4ca988c7ee3cda41591f2c20699c06826c89fe3 Aug 04 16:25:37.453778 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.453679 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" event={"ID":"eeca3752-e828-48ff-939c-ec5e1d83532c","Type":"ContainerStarted","Data":"b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f"} Aug 04 16:25:37.453778 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.453743 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" event={"ID":"eeca3752-e828-48ff-939c-ec5e1d83532c","Type":"ContainerStarted","Data":"2e51c65521097a9bc179aaf1d4ca988c7ee3cda41591f2c20699c06826c89fe3"} Aug 04 16:25:37.475330 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:37.474795 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" podStartSLOduration=1.4747744219999999 podStartE2EDuration="1.474774422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:25:37.472979472 +0000 UTC m=+1766.795402963" watchObservedRunningTime="2026-08-04 16:25:37.474774422 +0000 UTC m=+1766.797197903" Aug 04 16:25:47.196926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:47.196880 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:47.197340 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:47.196941 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:47.209779 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:47.209749 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:25:47.519362 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:25:47.519330 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:26:13.015064 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:13.015026 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx"] Aug 04 16:26:14.169632 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:14.169592 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7"] Aug 04 16:26:28.036014 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.035969 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" podUID="e58ac783-4909-4a44-8b2f-56353ac5202f" containerName="main" containerID="cri-o://ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670" gracePeriod=45 Aug 04 16:26:28.288822 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.288737 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:26:28.364583 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.364547 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e58ac783-4909-4a44-8b2f-56353ac5202f-tls-certs\") pod \"e58ac783-4909-4a44-8b2f-56353ac5202f\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " Aug 04 16:26:28.364805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.364611 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-tmp-dir\") pod \"e58ac783-4909-4a44-8b2f-56353ac5202f\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " Aug 04 16:26:28.364805 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.364750 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-dshm\") pod \"e58ac783-4909-4a44-8b2f-56353ac5202f\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " Aug 04 16:26:28.364940 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.364806 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72h6d\" (UniqueName: \"kubernetes.io/projected/e58ac783-4909-4a44-8b2f-56353ac5202f-kube-api-access-72h6d\") pod \"e58ac783-4909-4a44-8b2f-56353ac5202f\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " Aug 04 16:26:28.364940 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.364845 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-model-cache\") pod \"e58ac783-4909-4a44-8b2f-56353ac5202f\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " Aug 04 16:26:28.364940 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.364887 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-home\") pod \"e58ac783-4909-4a44-8b2f-56353ac5202f\" (UID: \"e58ac783-4909-4a44-8b2f-56353ac5202f\") " Aug 04 16:26:28.364940 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.364891 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e58ac783-4909-4a44-8b2f-56353ac5202f" (UID: "e58ac783-4909-4a44-8b2f-56353ac5202f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:26:28.365130 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.365094 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-model-cache" (OuterVolumeSpecName: "model-cache") pod "e58ac783-4909-4a44-8b2f-56353ac5202f" (UID: "e58ac783-4909-4a44-8b2f-56353ac5202f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:26:28.365176 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.365144 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-home" (OuterVolumeSpecName: "home") pod "e58ac783-4909-4a44-8b2f-56353ac5202f" (UID: "e58ac783-4909-4a44-8b2f-56353ac5202f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:26:28.365254 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.365238 2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-model-cache\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:28.365302 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.365261 2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-home\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:28.365302 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.365276 2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-tmp-dir\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:28.366960 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.366937 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e58ac783-4909-4a44-8b2f-56353ac5202f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e58ac783-4909-4a44-8b2f-56353ac5202f" (UID: "e58ac783-4909-4a44-8b2f-56353ac5202f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:26:28.367235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.367215 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e58ac783-4909-4a44-8b2f-56353ac5202f-kube-api-access-72h6d" (OuterVolumeSpecName: "kube-api-access-72h6d") pod "e58ac783-4909-4a44-8b2f-56353ac5202f" (UID: "e58ac783-4909-4a44-8b2f-56353ac5202f"). InnerVolumeSpecName "kube-api-access-72h6d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:26:28.367287 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.367233 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-dshm" (OuterVolumeSpecName: "dshm") pod "e58ac783-4909-4a44-8b2f-56353ac5202f" (UID: "e58ac783-4909-4a44-8b2f-56353ac5202f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:26:28.466685 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.466646 2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e58ac783-4909-4a44-8b2f-56353ac5202f-dshm\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:28.466685 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.466679 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72h6d\" (UniqueName: \"kubernetes.io/projected/e58ac783-4909-4a44-8b2f-56353ac5202f-kube-api-access-72h6d\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:28.466685 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.466690 2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e58ac783-4909-4a44-8b2f-56353ac5202f-tls-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:28.723470 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.723373 2556 generic.go:356] "Generic (PLEG): container finished" podID="e58ac783-4909-4a44-8b2f-56353ac5202f" containerID="ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670" exitCode=0 Aug 04 16:26:28.723470 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.723423 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" Aug 04 16:26:28.723681 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.723465 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" event={"ID":"e58ac783-4909-4a44-8b2f-56353ac5202f","Type":"ContainerDied","Data":"ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670"} Aug 04 16:26:28.723681 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.723512 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx" event={"ID":"e58ac783-4909-4a44-8b2f-56353ac5202f","Type":"ContainerDied","Data":"6e5e28b994fa34c037aaf2afbf04544dc4df04ab2a95e5f9c0928770cd1d9597"} Aug 04 16:26:28.723681 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.723535 2556 scope.go:117] "RemoveContainer" containerID="ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670" Aug 04 16:26:28.732566 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.732549 2556 scope.go:117] "RemoveContainer" containerID="ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670" Aug 04 16:26:28.732885 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:26:28.732866 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670\": container with ID starting with ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670 not found: ID does not exist" containerID="ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670" Aug 04 16:26:28.732951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.732898 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670"} err="failed to get container status \"ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670\": rpc error: code = NotFound desc = could not find container \"ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670\": container with ID starting with ad895c96a6a478c2902cc2d6e21edc99a24b59a6436365a8795dc2966b06a670 not found: ID does not exist" Aug 04 16:26:28.745376 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.745351 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx"] Aug 04 16:26:28.748756 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:28.748717 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-34eb1536-kserve-54474bcf79-88cnx"] Aug 04 16:26:29.190176 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.190128 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" podUID="eeca3752-e828-48ff-939c-ec5e1d83532c" containerName="main" containerID="cri-o://b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f" gracePeriod=45 Aug 04 16:26:29.273313 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.273276 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e58ac783-4909-4a44-8b2f-56353ac5202f" path="/var/lib/kubelet/pods/e58ac783-4909-4a44-8b2f-56353ac5202f/volumes" Aug 04 16:26:29.447862 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.447788 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:26:29.475825 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.475796 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeca3752-e828-48ff-939c-ec5e1d83532c-tls-certs\") pod \"eeca3752-e828-48ff-939c-ec5e1d83532c\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " Aug 04 16:26:29.475995 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.475834 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-tmp-dir\") pod \"eeca3752-e828-48ff-939c-ec5e1d83532c\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " Aug 04 16:26:29.475995 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.475880 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45h8g\" (UniqueName: \"kubernetes.io/projected/eeca3752-e828-48ff-939c-ec5e1d83532c-kube-api-access-45h8g\") pod \"eeca3752-e828-48ff-939c-ec5e1d83532c\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " Aug 04 16:26:29.475995 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.475898 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-dshm\") pod \"eeca3752-e828-48ff-939c-ec5e1d83532c\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " Aug 04 16:26:29.475995 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.475988 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-model-cache\") pod \"eeca3752-e828-48ff-939c-ec5e1d83532c\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " Aug 04 16:26:29.476222 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.476015 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-home\") pod \"eeca3752-e828-48ff-939c-ec5e1d83532c\" (UID: \"eeca3752-e828-48ff-939c-ec5e1d83532c\") " Aug 04 16:26:29.476222 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.476122 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eeca3752-e828-48ff-939c-ec5e1d83532c" (UID: "eeca3752-e828-48ff-939c-ec5e1d83532c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:26:29.476325 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.476247 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-model-cache" (OuterVolumeSpecName: "model-cache") pod "eeca3752-e828-48ff-939c-ec5e1d83532c" (UID: "eeca3752-e828-48ff-939c-ec5e1d83532c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:26:29.476325 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.476316 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-home" (OuterVolumeSpecName: "home") pod "eeca3752-e828-48ff-939c-ec5e1d83532c" (UID: "eeca3752-e828-48ff-939c-ec5e1d83532c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:26:29.476419 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.476383 2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-model-cache\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:29.476419 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.476396 2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-tmp-dir\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:29.478272 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.478244 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-dshm" (OuterVolumeSpecName: "dshm") pod "eeca3752-e828-48ff-939c-ec5e1d83532c" (UID: "eeca3752-e828-48ff-939c-ec5e1d83532c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:26:29.478272 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.478267 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eeca3752-e828-48ff-939c-ec5e1d83532c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eeca3752-e828-48ff-939c-ec5e1d83532c" (UID: "eeca3752-e828-48ff-939c-ec5e1d83532c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:26:29.478462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.478445 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeca3752-e828-48ff-939c-ec5e1d83532c-kube-api-access-45h8g" (OuterVolumeSpecName: "kube-api-access-45h8g") pod "eeca3752-e828-48ff-939c-ec5e1d83532c" (UID: "eeca3752-e828-48ff-939c-ec5e1d83532c"). InnerVolumeSpecName "kube-api-access-45h8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:26:29.577544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.577507 2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-home\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:29.577544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.577536 2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeca3752-e828-48ff-939c-ec5e1d83532c-tls-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:29.577544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.577547 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45h8g\" (UniqueName: \"kubernetes.io/projected/eeca3752-e828-48ff-939c-ec5e1d83532c-kube-api-access-45h8g\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:29.577544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.577555 2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eeca3752-e828-48ff-939c-ec5e1d83532c-dshm\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:26:29.730203 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.730106 2556 generic.go:356] "Generic (PLEG): container finished" podID="eeca3752-e828-48ff-939c-ec5e1d83532c" containerID="b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f" exitCode=0 Aug 04 16:26:29.730203 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.730156 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" Aug 04 16:26:29.730447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.730203 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" event={"ID":"eeca3752-e828-48ff-939c-ec5e1d83532c","Type":"ContainerDied","Data":"b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f"} Aug 04 16:26:29.730447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.730260 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7" event={"ID":"eeca3752-e828-48ff-939c-ec5e1d83532c","Type":"ContainerDied","Data":"2e51c65521097a9bc179aaf1d4ca988c7ee3cda41591f2c20699c06826c89fe3"} Aug 04 16:26:29.730447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.730284 2556 scope.go:117] "RemoveContainer" containerID="b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f" Aug 04 16:26:29.739354 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.739336 2556 scope.go:117] "RemoveContainer" containerID="b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f" Aug 04 16:26:29.739627 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:26:29.739608 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f\": container with ID starting with b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f not found: ID does not exist" containerID="b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f" Aug 04 16:26:29.739678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.739637 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f"} err="failed to get container status \"b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f\": rpc error: code = NotFound desc = could not find container \"b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f\": container with ID starting with b48cc8476f0de06686fd9db19a9c1a6be1dae5e6c4640e9497756064ebe8cb7f not found: ID does not exist" Aug 04 16:26:29.753986 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.753957 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7"] Aug 04 16:26:29.757358 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:29.757334 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-34eb1536-kserve-58f5c8df87-579g7"] Aug 04 16:26:31.280198 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:26:31.280160 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeca3752-e828-48ff-939c-ec5e1d83532c" path="/var/lib/kubelet/pods/eeca3752-e828-48ff-939c-ec5e1d83532c/volumes" Aug 04 16:30:00.136942 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.136900 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87"] Aug 04 16:30:00.137461 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.137332 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeca3752-e828-48ff-939c-ec5e1d83532c" containerName="main" Aug 04 16:30:00.137461 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.137353 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeca3752-e828-48ff-939c-ec5e1d83532c" containerName="main" Aug 04 16:30:00.137461 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.137362 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e58ac783-4909-4a44-8b2f-56353ac5202f" containerName="main" Aug 04 16:30:00.137461 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.137369 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e58ac783-4909-4a44-8b2f-56353ac5202f" containerName="main" Aug 04 16:30:00.137461 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.137438 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeca3752-e828-48ff-939c-ec5e1d83532c" containerName="main" Aug 04 16:30:00.137461 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.137455 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e58ac783-4909-4a44-8b2f-56353ac5202f" containerName="main" Aug 04 16:30:00.140807 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.140783 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87" Aug 04 16:30:00.143671 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.143646 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-psz5k\"" Aug 04 16:30:00.150384 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.150363 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87" Aug 04 16:30:00.153995 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.153967 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87"] Aug 04 16:30:00.285433 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.285405 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87"] Aug 04 16:30:00.287022 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:30:00.286998 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d587de_1a08_4921_8635_b4a4c02cd963.slice/crio-c5a21c482f32a08f9f0f4a595776e1f2d5957875911658d4134b8c85d739eb0b WatchSource:0}: Error finding container c5a21c482f32a08f9f0f4a595776e1f2d5957875911658d4134b8c85d739eb0b: Status 404 returned error can't find the container with id c5a21c482f32a08f9f0f4a595776e1f2d5957875911658d4134b8c85d739eb0b Aug 04 16:30:00.791852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.791768 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87" event={"ID":"12d587de-1a08-4921-8635-b4a4c02cd963","Type":"ContainerStarted","Data":"f4f70833335e58258b0e07821858fc5d6a3416a764511579e516d67add256f33"} Aug 04 16:30:00.791852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.791850 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87" event={"ID":"12d587de-1a08-4921-8635-b4a4c02cd963","Type":"ContainerStarted","Data":"c5a21c482f32a08f9f0f4a595776e1f2d5957875911658d4134b8c85d739eb0b"} Aug 04 16:30:00.809345 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:00.809289 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87" podStartSLOduration=0.809274299 podStartE2EDuration="809.274299ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:30:00.807012042 +0000 UTC m=+2030.129435533" watchObservedRunningTime="2026-08-04 16:30:00.809274299 +0000 UTC m=+2030.131697779" Aug 04 16:30:01.797987 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:01.797942 2556 generic.go:356] "Generic (PLEG): container finished" podID="12d587de-1a08-4921-8635-b4a4c02cd963" containerID="f4f70833335e58258b0e07821858fc5d6a3416a764511579e516d67add256f33" exitCode=0 Aug 04 16:30:01.798445 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:01.798030 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87" event={"ID":"12d587de-1a08-4921-8635-b4a4c02cd963","Type":"ContainerDied","Data":"f4f70833335e58258b0e07821858fc5d6a3416a764511579e516d67add256f33"} Aug 04 16:30:02.939530 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:02.939501 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87" Aug 04 16:30:03.809111 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:03.809082 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87" Aug 04 16:30:03.809282 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:03.809128 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764350-g4g87" event={"ID":"12d587de-1a08-4921-8635-b4a4c02cd963","Type":"ContainerDied","Data":"c5a21c482f32a08f9f0f4a595776e1f2d5957875911658d4134b8c85d739eb0b"} Aug 04 16:30:03.809282 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:03.809177 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5a21c482f32a08f9f0f4a595776e1f2d5957875911658d4134b8c85d739eb0b" Aug 04 16:30:09.776950 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.776907 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f"] Aug 04 16:30:09.777646 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.777532 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12d587de-1a08-4921-8635-b4a4c02cd963" containerName="cleanup" Aug 04 16:30:09.777815 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.777649 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="12d587de-1a08-4921-8635-b4a4c02cd963" containerName="cleanup" Aug 04 16:30:09.777815 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.777751 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="12d587de-1a08-4921-8635-b4a4c02cd963" containerName="cleanup" Aug 04 16:30:09.781464 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.781430 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f"] Aug 04 16:30:09.781604 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.781556 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.784278 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.784249 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3a345e2e-openshift-default-dockercfg-k7trr\"" Aug 04 16:30:09.848131 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.848088 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-envoy\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.848328 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.848138 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-certs\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.848328 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.848176 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2340311e-52f8-4b01-a04b-738278097627-istio-podinfo\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.848328 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.848214 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2340311e-52f8-4b01-a04b-738278097627-istiod-ca-cert\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.848328 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.848240 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhrhc\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-kube-api-access-nhrhc\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.848481 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.848358 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-istio-token\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.848481 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.848417 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-socket\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.848545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.848500 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-data\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.848545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.848533 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-credential-socket\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.949347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949310 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2340311e-52f8-4b01-a04b-738278097627-istiod-ca-cert\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.949347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949352 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhrhc\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-kube-api-access-nhrhc\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.949636 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949385 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-istio-token\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.949636 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949427 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-socket\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.949636 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949459 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-data\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.949636 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949491 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-credential-socket\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.949636 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949555 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-envoy\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.949636 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949586 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-certs\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.949636 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949618 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2340311e-52f8-4b01-a04b-738278097627-istio-podinfo\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.950040 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.949960 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-data\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.950040 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.950021 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-socket\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.950151 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.950053 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-credential-socket\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.950229 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.950205 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-certs\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.950460 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.950438 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2340311e-52f8-4b01-a04b-738278097627-istiod-ca-cert\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.952020 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.951992 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-envoy\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.952318 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.952298 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2340311e-52f8-4b01-a04b-738278097627-istio-podinfo\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.958610 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.958581 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhrhc\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-kube-api-access-nhrhc\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:09.958716 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:09.958652 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-istio-token\") pod \"e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:10.094289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:10.094197 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:10.239707 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:10.239673 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f"] Aug 04 16:30:10.241127 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:30:10.241101 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2340311e_52f8_4b01_a04b_738278097627.slice/crio-b536e65af064c5d5ee5b127d76412598f95c674e7e182827ee6210e4cf220ec3 WatchSource:0}: Error finding container b536e65af064c5d5ee5b127d76412598f95c674e7e182827ee6210e4cf220ec3: Status 404 returned error can't find the container with id b536e65af064c5d5ee5b127d76412598f95c674e7e182827ee6210e4cf220ec3 Aug 04 16:30:10.243938 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:10.243903 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:30:10.244021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:10.243979 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:30:10.244021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:10.244009 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:30:10.847529 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:10.847491 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" event={"ID":"2340311e-52f8-4b01-a04b-738278097627","Type":"ContainerStarted","Data":"7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3"} Aug 04 16:30:10.847966 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:10.847546 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" event={"ID":"2340311e-52f8-4b01-a04b-738278097627","Type":"ContainerStarted","Data":"b536e65af064c5d5ee5b127d76412598f95c674e7e182827ee6210e4cf220ec3"} Aug 04 16:30:10.873682 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:10.873628 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" podStartSLOduration=1.873609461 podStartE2EDuration="1.873609461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:30:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:30:10.87114967 +0000 UTC m=+2040.193573152" watchObservedRunningTime="2026-08-04 16:30:10.873609461 +0000 UTC m=+2040.196032945" Aug 04 16:30:11.094473 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:11.094436 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:11.099531 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:11.099465 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:11.853149 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:11.853122 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:11.854260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:11.854236 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:14.191187 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.191148 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg"] Aug 04 16:30:14.194990 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.194968 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:14.200012 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.199991 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3a345e2e-dockercfg-rxl8q\"" Aug 04 16:30:14.200255 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.200239 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3a345e2e\"" Aug 04 16:30:14.210163 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.210135 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg"] Aug 04 16:30:14.221456 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.221421 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs"] Aug 04 16:30:14.225345 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.225319 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:14.239700 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.239668 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs"] Aug 04 16:30:14.293345 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.293300 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f9907e-91f6-4c13-bc30-a61cb9b759f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg\" (UID: \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:14.293536 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.293356 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776244f2-4578-4be6-a236-c3921fac9fbd-plugins-config-volume\") pod \"payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs\" (UID: \"776244f2-4578-4be6-a236-c3921fac9fbd\") " pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:14.293536 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.293419 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk5dc\" (UniqueName: \"kubernetes.io/projected/42f9907e-91f6-4c13-bc30-a61cb9b759f2-kube-api-access-qk5dc\") pod \"payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg\" (UID: \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:14.293536 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.293487 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sjts\" (UniqueName: \"kubernetes.io/projected/776244f2-4578-4be6-a236-c3921fac9fbd-kube-api-access-6sjts\") pod \"payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs\" (UID: \"776244f2-4578-4be6-a236-c3921fac9fbd\") " pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:14.394654 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.394619 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sjts\" (UniqueName: \"kubernetes.io/projected/776244f2-4578-4be6-a236-c3921fac9fbd-kube-api-access-6sjts\") pod \"payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs\" (UID: \"776244f2-4578-4be6-a236-c3921fac9fbd\") " pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:14.394846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.394714 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f9907e-91f6-4c13-bc30-a61cb9b759f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg\" (UID: \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:14.394846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.394796 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776244f2-4578-4be6-a236-c3921fac9fbd-plugins-config-volume\") pod \"payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs\" (UID: \"776244f2-4578-4be6-a236-c3921fac9fbd\") " pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:14.394846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.394827 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk5dc\" (UniqueName: \"kubernetes.io/projected/42f9907e-91f6-4c13-bc30-a61cb9b759f2-kube-api-access-qk5dc\") pod \"payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg\" (UID: \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:14.395544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.395524 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776244f2-4578-4be6-a236-c3921fac9fbd-plugins-config-volume\") pod \"payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs\" (UID: \"776244f2-4578-4be6-a236-c3921fac9fbd\") " pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:14.395587 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.395527 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f9907e-91f6-4c13-bc30-a61cb9b759f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg\" (UID: \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:14.403890 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.403859 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk5dc\" (UniqueName: \"kubernetes.io/projected/42f9907e-91f6-4c13-bc30-a61cb9b759f2-kube-api-access-qk5dc\") pod \"payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg\" (UID: \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:14.404023 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.403934 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sjts\" (UniqueName: \"kubernetes.io/projected/776244f2-4578-4be6-a236-c3921fac9fbd-kube-api-access-6sjts\") pod \"payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs\" (UID: \"776244f2-4578-4be6-a236-c3921fac9fbd\") " pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:14.505219 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.505181 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:14.536656 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.536617 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:14.663412 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.663379 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg"] Aug 04 16:30:14.694846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.694774 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs"] Aug 04 16:30:14.701988 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:30:14.701960 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776244f2_4578_4be6_a236_c3921fac9fbd.slice/crio-856f84aaaf892fb99da90fe6f2bc46e3164cd25bd5874d02abe174450b980bbb WatchSource:0}: Error finding container 856f84aaaf892fb99da90fe6f2bc46e3164cd25bd5874d02abe174450b980bbb: Status 404 returned error can't find the container with id 856f84aaaf892fb99da90fe6f2bc46e3164cd25bd5874d02abe174450b980bbb Aug 04 16:30:14.872571 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.872470 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" event={"ID":"776244f2-4578-4be6-a236-c3921fac9fbd","Type":"ContainerStarted","Data":"51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d"} Aug 04 16:30:14.872571 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.872539 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:14.872833 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.872597 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" event={"ID":"776244f2-4578-4be6-a236-c3921fac9fbd","Type":"ContainerStarted","Data":"856f84aaaf892fb99da90fe6f2bc46e3164cd25bd5874d02abe174450b980bbb"} Aug 04 16:30:14.874196 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.874166 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" event={"ID":"42f9907e-91f6-4c13-bc30-a61cb9b759f2","Type":"ContainerStarted","Data":"303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5"} Aug 04 16:30:14.874318 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.874214 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" event={"ID":"42f9907e-91f6-4c13-bc30-a61cb9b759f2","Type":"ContainerStarted","Data":"7dedf9b89e4ac84206b03f87977ec89076c180c7416068c6434eee58089aa7b0"} Aug 04 16:30:14.874318 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.874286 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:14.889935 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.889844 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" podStartSLOduration=0.88980591 podStartE2EDuration="889.80591ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:30:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:30:14.889744022 +0000 UTC m=+2044.212167505" watchObservedRunningTime="2026-08-04 16:30:14.88980591 +0000 UTC m=+2044.212229389" Aug 04 16:30:14.907639 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:14.907586 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" podStartSLOduration=0.907568201 podStartE2EDuration="907.568201ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:30:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:30:14.906861169 +0000 UTC m=+2044.229284651" watchObservedRunningTime="2026-08-04 16:30:14.907568201 +0000 UTC m=+2044.229991716" Aug 04 16:30:25.410885 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:30:25.410852 2556 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2340311e_52f8_4b01_a04b_738278097627.slice/crio-7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3.scope\": RecentStats: unable to find data in memory cache]" Aug 04 16:30:25.882956 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:25.882913 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:25.883123 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:25.883012 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:25.944562 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:25.944529 2556 generic.go:356] "Generic (PLEG): container finished" podID="2340311e-52f8-4b01-a04b-738278097627" containerID="7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3" exitCode=0 Aug 04 16:30:25.944755 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:25.944599 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" event={"ID":"2340311e-52f8-4b01-a04b-738278097627","Type":"ContainerDied","Data":"7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3"} Aug 04 16:30:25.945067 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:25.945051 2556 scope.go:117] "RemoveContainer" containerID="7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3" Aug 04 16:30:25.946269 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:25.946232 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:30:25.946357 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:25.946326 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:30:25.946400 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:25.946358 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:30:26.853957 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:26.853924 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:26.952202 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:26.952158 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" event={"ID":"2340311e-52f8-4b01-a04b-738278097627","Type":"ContainerStarted","Data":"1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11"} Aug 04 16:30:27.095517 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:27.095462 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:27.096912 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:27.096877 2556 patch_prober.go:28] interesting pod/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Aug 04 16:30:27.097052 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:27.096945 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Aug 04 16:30:28.095808 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:28.095765 2556 patch_prober.go:28] interesting pod/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Aug 04 16:30:28.096449 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:28.096413 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Aug 04 16:30:29.096126 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:29.095634 2556 patch_prober.go:28] interesting pod/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Aug 04 16:30:29.096126 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:29.095718 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Aug 04 16:30:30.094792 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:30.094750 2556 patch_prober.go:28] interesting pod/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Aug 04 16:30:30.095108 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:30.095077 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Aug 04 16:30:31.095333 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:31.094865 2556 patch_prober.go:28] interesting pod/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Aug 04 16:30:31.095333 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:31.094937 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Aug 04 16:30:32.094803 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:32.094758 2556 patch_prober.go:28] interesting pod/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Aug 04 16:30:32.094992 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:32.094836 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Aug 04 16:30:33.098982 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:33.098949 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:33.099385 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:33.099192 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:33.099967 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:33.099946 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:53.993877 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:53.993831 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs"] Aug 04 16:30:53.994444 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:53.994303 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" podUID="776244f2-4578-4be6-a236-c3921fac9fbd" containerName="payload-processing" containerID="cri-o://51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d" gracePeriod=30 Aug 04 16:30:54.002519 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.002490 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg"] Aug 04 16:30:54.003289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.002759 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" podUID="42f9907e-91f6-4c13-bc30-a61cb9b759f2" containerName="payload-pre-processing" containerID="cri-o://303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5" gracePeriod=30 Aug 04 16:30:54.268066 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.267758 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:54.297002 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.296970 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:54.302514 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.302474 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f"] Aug 04 16:30:54.302760 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.302715 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" containerID="cri-o://1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11" gracePeriod=30 Aug 04 16:30:54.395092 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.393138 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776244f2-4578-4be6-a236-c3921fac9fbd-plugins-config-volume\") pod \"776244f2-4578-4be6-a236-c3921fac9fbd\" (UID: \"776244f2-4578-4be6-a236-c3921fac9fbd\") " Aug 04 16:30:54.395092 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.393197 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk5dc\" (UniqueName: \"kubernetes.io/projected/42f9907e-91f6-4c13-bc30-a61cb9b759f2-kube-api-access-qk5dc\") pod \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\" (UID: \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\") " Aug 04 16:30:54.395092 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.393252 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sjts\" (UniqueName: \"kubernetes.io/projected/776244f2-4578-4be6-a236-c3921fac9fbd-kube-api-access-6sjts\") pod \"776244f2-4578-4be6-a236-c3921fac9fbd\" (UID: \"776244f2-4578-4be6-a236-c3921fac9fbd\") " Aug 04 16:30:54.395092 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.393381 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f9907e-91f6-4c13-bc30-a61cb9b759f2-plugins-config-volume\") pod \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\" (UID: \"42f9907e-91f6-4c13-bc30-a61cb9b759f2\") " Aug 04 16:30:54.395092 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.394327 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42f9907e-91f6-4c13-bc30-a61cb9b759f2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "42f9907e-91f6-4c13-bc30-a61cb9b759f2" (UID: "42f9907e-91f6-4c13-bc30-a61cb9b759f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:30:54.395092 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.394747 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/776244f2-4578-4be6-a236-c3921fac9fbd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "776244f2-4578-4be6-a236-c3921fac9fbd" (UID: "776244f2-4578-4be6-a236-c3921fac9fbd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:30:54.400620 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.400555 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/776244f2-4578-4be6-a236-c3921fac9fbd-kube-api-access-6sjts" (OuterVolumeSpecName: "kube-api-access-6sjts") pod "776244f2-4578-4be6-a236-c3921fac9fbd" (UID: "776244f2-4578-4be6-a236-c3921fac9fbd"). InnerVolumeSpecName "kube-api-access-6sjts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:30:54.402213 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.402173 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f9907e-91f6-4c13-bc30-a61cb9b759f2-kube-api-access-qk5dc" (OuterVolumeSpecName: "kube-api-access-qk5dc") pod "42f9907e-91f6-4c13-bc30-a61cb9b759f2" (UID: "42f9907e-91f6-4c13-bc30-a61cb9b759f2"). InnerVolumeSpecName "kube-api-access-qk5dc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:30:54.494944 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.494906 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sjts\" (UniqueName: \"kubernetes.io/projected/776244f2-4578-4be6-a236-c3921fac9fbd-kube-api-access-6sjts\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:54.494944 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.494937 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f9907e-91f6-4c13-bc30-a61cb9b759f2-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:54.494944 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.494946 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776244f2-4578-4be6-a236-c3921fac9fbd-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:54.494944 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:54.494955 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk5dc\" (UniqueName: \"kubernetes.io/projected/42f9907e-91f6-4c13-bc30-a61cb9b759f2-kube-api-access-qk5dc\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:55.100148 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.100108 2556 generic.go:356] "Generic (PLEG): container finished" podID="42f9907e-91f6-4c13-bc30-a61cb9b759f2" containerID="303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5" exitCode=0 Aug 04 16:30:55.100587 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.100164 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" Aug 04 16:30:55.100587 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.100197 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" event={"ID":"42f9907e-91f6-4c13-bc30-a61cb9b759f2","Type":"ContainerDied","Data":"303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5"} Aug 04 16:30:55.100587 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.100289 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg" event={"ID":"42f9907e-91f6-4c13-bc30-a61cb9b759f2","Type":"ContainerDied","Data":"7dedf9b89e4ac84206b03f87977ec89076c180c7416068c6434eee58089aa7b0"} Aug 04 16:30:55.100587 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.100318 2556 scope.go:117] "RemoveContainer" containerID="303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5" Aug 04 16:30:55.104616 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.104586 2556 generic.go:356] "Generic (PLEG): container finished" podID="776244f2-4578-4be6-a236-c3921fac9fbd" containerID="51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d" exitCode=0 Aug 04 16:30:55.104782 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.104656 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" event={"ID":"776244f2-4578-4be6-a236-c3921fac9fbd","Type":"ContainerDied","Data":"51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d"} Aug 04 16:30:55.104782 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.104665 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" Aug 04 16:30:55.104782 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.104700 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs" event={"ID":"776244f2-4578-4be6-a236-c3921fac9fbd","Type":"ContainerDied","Data":"856f84aaaf892fb99da90fe6f2bc46e3164cd25bd5874d02abe174450b980bbb"} Aug 04 16:30:55.113599 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.113581 2556 scope.go:117] "RemoveContainer" containerID="303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5" Aug 04 16:30:55.113931 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:30:55.113911 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5\": container with ID starting with 303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5 not found: ID does not exist" containerID="303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5" Aug 04 16:30:55.113987 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.113942 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5"} err="failed to get container status \"303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5\": rpc error: code = NotFound desc = could not find container \"303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5\": container with ID starting with 303f84c4735c39200d2f06ce1e5a59366c7dd0e20bf7c430c66e30d84eba6ed5 not found: ID does not exist" Aug 04 16:30:55.113987 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.113965 2556 scope.go:117] "RemoveContainer" containerID="51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d" Aug 04 16:30:55.121705 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.121685 2556 scope.go:117] "RemoveContainer" containerID="51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d" Aug 04 16:30:55.121959 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:30:55.121941 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d\": container with ID starting with 51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d not found: ID does not exist" containerID="51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d" Aug 04 16:30:55.122018 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.121964 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d"} err="failed to get container status \"51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d\": rpc error: code = NotFound desc = could not find container \"51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d\": container with ID starting with 51ad3ca842bbd9a3b93d8a662871938f979d8e28d420937018c2be683553d21d not found: ID does not exist" Aug 04 16:30:55.137731 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.137701 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg"] Aug 04 16:30:55.141410 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.141389 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3a345e2e-67c45fb5b8-wjcfg"] Aug 04 16:30:55.177305 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.177278 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs"] Aug 04 16:30:55.189588 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.189564 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3a345e2e-5777dd6f6-hbqqs"] Aug 04 16:30:55.273302 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.273221 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42f9907e-91f6-4c13-bc30-a61cb9b759f2" path="/var/lib/kubelet/pods/42f9907e-91f6-4c13-bc30-a61cb9b759f2/volumes" Aug 04 16:30:55.273600 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.273587 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="776244f2-4578-4be6-a236-c3921fac9fbd" path="/var/lib/kubelet/pods/776244f2-4578-4be6-a236-c3921fac9fbd/volumes" Aug 04 16:30:55.361874 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.361842 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995"] Aug 04 16:30:55.362291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.362273 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f9907e-91f6-4c13-bc30-a61cb9b759f2" containerName="payload-pre-processing" Aug 04 16:30:55.362291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.362291 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f9907e-91f6-4c13-bc30-a61cb9b759f2" containerName="payload-pre-processing" Aug 04 16:30:55.362440 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.362331 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="776244f2-4578-4be6-a236-c3921fac9fbd" containerName="payload-processing" Aug 04 16:30:55.362440 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.362341 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="776244f2-4578-4be6-a236-c3921fac9fbd" containerName="payload-processing" Aug 04 16:30:55.362440 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.362411 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="776244f2-4578-4be6-a236-c3921fac9fbd" containerName="payload-processing" Aug 04 16:30:55.362440 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.362423 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f9907e-91f6-4c13-bc30-a61cb9b759f2" containerName="payload-pre-processing" Aug 04 16:30:55.367306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.367287 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.369989 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.369966 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-64639142-gw-openshift-default-dockercfg-mm4hj\"" Aug 04 16:30:55.379485 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.379461 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995"] Aug 04 16:30:55.503750 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.503687 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-podinfo\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.503939 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.503795 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-certs\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.503939 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.503826 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-envoy\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.503939 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.503853 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-data\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.503939 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.503889 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-socket\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.503939 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.503908 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-credential-socket\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.503939 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.503936 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttmlj\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-kube-api-access-ttmlj\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.504140 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.503954 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istiod-ca-cert\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.504140 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.503981 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-token\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.604997 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.604888 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-certs\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.604997 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.604945 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-envoy\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.604997 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.604971 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-data\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.604997 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.604990 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-socket\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.605347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605011 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-credential-socket\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.605347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605144 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttmlj\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-kube-api-access-ttmlj\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.605347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605208 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istiod-ca-cert\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.605347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605247 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-token\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.605347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605326 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-certs\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.605347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605342 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-podinfo\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.605628 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605421 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-socket\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.605628 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605608 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-data\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.605757 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605697 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-credential-socket\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.606008 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.605985 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istiod-ca-cert\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.607531 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.607508 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-envoy\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.607781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.607764 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-podinfo\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.613896 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.613875 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttmlj\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-kube-api-access-ttmlj\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.613990 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.613923 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-token\") pod \"e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.677754 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.677692 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:55.821521 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.821496 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995"] Aug 04 16:30:55.823264 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:30:55.823238 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052e0fa9_7e2e_4bb4_aa0c_d484c0338025.slice/crio-1703774a6692525f6a7159ef26b61d96421dfbf6d81d8ae25396c0db793e8f2d WatchSource:0}: Error finding container 1703774a6692525f6a7159ef26b61d96421dfbf6d81d8ae25396c0db793e8f2d: Status 404 returned error can't find the container with id 1703774a6692525f6a7159ef26b61d96421dfbf6d81d8ae25396c0db793e8f2d Aug 04 16:30:55.825477 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.825446 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:30:55.825544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.825512 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:30:55.825581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:55.825541 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:30:56.112219 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:56.112184 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" event={"ID":"052e0fa9-7e2e-4bb4-aa0c-d484c0338025","Type":"ContainerStarted","Data":"914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2"} Aug 04 16:30:56.112650 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:56.112244 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" event={"ID":"052e0fa9-7e2e-4bb4-aa0c-d484c0338025","Type":"ContainerStarted","Data":"1703774a6692525f6a7159ef26b61d96421dfbf6d81d8ae25396c0db793e8f2d"} Aug 04 16:30:56.133302 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:56.133206 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" podStartSLOduration=1.1331884620000001 podStartE2EDuration="1.133188462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:30:56.131515364 +0000 UTC m=+2085.453938871" watchObservedRunningTime="2026-08-04 16:30:56.133188462 +0000 UTC m=+2085.455611941" Aug 04 16:30:56.677898 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:56.677862 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:56.683391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:56.683365 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:57.120017 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:57.119979 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:57.120982 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:57.120960 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:30:58.581959 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.581922 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q"] Aug 04 16:30:58.585827 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.585809 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:30:58.588928 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.588903 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-64639142\"" Aug 04 16:30:58.589142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.589117 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-64639142-dockercfg-nvshn\"" Aug 04 16:30:58.594681 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.594660 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q"] Aug 04 16:30:58.636333 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.636284 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d979x\" (UniqueName: \"kubernetes.io/projected/61f3cdb7-8179-425a-b870-84b8ec00f293-kube-api-access-d979x\") pod \"payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q\" (UID: \"61f3cdb7-8179-425a-b870-84b8ec00f293\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:30:58.636554 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.636525 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61f3cdb7-8179-425a-b870-84b8ec00f293-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q\" (UID: \"61f3cdb7-8179-425a-b870-84b8ec00f293\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:30:58.645841 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.645800 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn"] Aug 04 16:30:58.650146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.650118 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:30:58.661148 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.661113 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn"] Aug 04 16:30:58.737639 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.737595 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktztc\" (UniqueName: \"kubernetes.io/projected/dd0c7e98-91be-4ed4-8132-885c8ddd812f-kube-api-access-ktztc\") pod \"payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn\" (UID: \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\") " pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:30:58.737833 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.737656 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61f3cdb7-8179-425a-b870-84b8ec00f293-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q\" (UID: \"61f3cdb7-8179-425a-b870-84b8ec00f293\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:30:58.737833 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.737770 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d979x\" (UniqueName: \"kubernetes.io/projected/61f3cdb7-8179-425a-b870-84b8ec00f293-kube-api-access-d979x\") pod \"payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q\" (UID: \"61f3cdb7-8179-425a-b870-84b8ec00f293\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:30:58.737972 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.737828 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0c7e98-91be-4ed4-8132-885c8ddd812f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn\" (UID: \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\") " pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:30:58.738448 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.738427 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61f3cdb7-8179-425a-b870-84b8ec00f293-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q\" (UID: \"61f3cdb7-8179-425a-b870-84b8ec00f293\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:30:58.746297 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.746268 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d979x\" (UniqueName: \"kubernetes.io/projected/61f3cdb7-8179-425a-b870-84b8ec00f293-kube-api-access-d979x\") pod \"payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q\" (UID: \"61f3cdb7-8179-425a-b870-84b8ec00f293\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:30:58.839488 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.839389 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0c7e98-91be-4ed4-8132-885c8ddd812f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn\" (UID: \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\") " pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:30:58.839488 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.839477 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktztc\" (UniqueName: \"kubernetes.io/projected/dd0c7e98-91be-4ed4-8132-885c8ddd812f-kube-api-access-ktztc\") pod \"payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn\" (UID: \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\") " pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:30:58.840248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.840223 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0c7e98-91be-4ed4-8132-885c8ddd812f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn\" (UID: \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\") " pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:30:58.848185 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.848159 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktztc\" (UniqueName: \"kubernetes.io/projected/dd0c7e98-91be-4ed4-8132-885c8ddd812f-kube-api-access-ktztc\") pod \"payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn\" (UID: \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\") " pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:30:58.898164 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.898127 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:30:58.962143 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:58.962113 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:30:59.051181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.051146 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q"] Aug 04 16:30:59.051509 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:30:59.051475 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f3cdb7_8179_425a_b870_84b8ec00f293.slice/crio-6164606f6368708afa0adea7ab5525a11bbb57d9ad17111a046b829ade7ab4b5 WatchSource:0}: Error finding container 6164606f6368708afa0adea7ab5525a11bbb57d9ad17111a046b829ade7ab4b5: Status 404 returned error can't find the container with id 6164606f6368708afa0adea7ab5525a11bbb57d9ad17111a046b829ade7ab4b5 Aug 04 16:30:59.132416 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.132388 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" event={"ID":"61f3cdb7-8179-425a-b870-84b8ec00f293","Type":"ContainerStarted","Data":"6164606f6368708afa0adea7ab5525a11bbb57d9ad17111a046b829ade7ab4b5"} Aug 04 16:30:59.327531 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.327502 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn"] Aug 04 16:30:59.328398 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:30:59.328373 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0c7e98_91be_4ed4_8132_885c8ddd812f.slice/crio-839b757c99687947025d6795aa7af12251988e6b3aa2acbd21ab95fe212b28f3 WatchSource:0}: Error finding container 839b757c99687947025d6795aa7af12251988e6b3aa2acbd21ab95fe212b28f3: Status 404 returned error can't find the container with id 839b757c99687947025d6795aa7af12251988e6b3aa2acbd21ab95fe212b28f3 Aug 04 16:30:59.556873 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.556849 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:30:59.648387 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648305 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2340311e-52f8-4b01-a04b-738278097627-istio-podinfo\") pod \"2340311e-52f8-4b01-a04b-738278097627\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " Aug 04 16:30:59.648387 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648371 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-istio-token\") pod \"2340311e-52f8-4b01-a04b-738278097627\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648469 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-credential-socket\") pod \"2340311e-52f8-4b01-a04b-738278097627\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648511 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-data\") pod \"2340311e-52f8-4b01-a04b-738278097627\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648541 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-socket\") pod \"2340311e-52f8-4b01-a04b-738278097627\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648653 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-envoy\") pod \"2340311e-52f8-4b01-a04b-738278097627\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648693 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-certs\") pod \"2340311e-52f8-4b01-a04b-738278097627\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648699 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2340311e-52f8-4b01-a04b-738278097627" (UID: "2340311e-52f8-4b01-a04b-738278097627"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648715 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-data" (OuterVolumeSpecName: "istio-data") pod "2340311e-52f8-4b01-a04b-738278097627" (UID: "2340311e-52f8-4b01-a04b-738278097627"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648719 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhrhc\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-kube-api-access-nhrhc\") pod \"2340311e-52f8-4b01-a04b-738278097627\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648785 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2340311e-52f8-4b01-a04b-738278097627-istiod-ca-cert\") pod \"2340311e-52f8-4b01-a04b-738278097627\" (UID: \"2340311e-52f8-4b01-a04b-738278097627\") " Aug 04 16:30:59.648901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.648845 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2340311e-52f8-4b01-a04b-738278097627" (UID: "2340311e-52f8-4b01-a04b-738278097627"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:30:59.649391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.649002 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2340311e-52f8-4b01-a04b-738278097627" (UID: "2340311e-52f8-4b01-a04b-738278097627"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:30:59.649391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.649129 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2340311e-52f8-4b01-a04b-738278097627-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2340311e-52f8-4b01-a04b-738278097627" (UID: "2340311e-52f8-4b01-a04b-738278097627"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:30:59.649391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.649211 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:59.649391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.649231 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:59.649391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.649246 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:59.649391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.649260 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:59.650828 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.650797 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2340311e-52f8-4b01-a04b-738278097627" (UID: "2340311e-52f8-4b01-a04b-738278097627"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:30:59.650828 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.650811 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-istio-token" (OuterVolumeSpecName: "istio-token") pod "2340311e-52f8-4b01-a04b-738278097627" (UID: "2340311e-52f8-4b01-a04b-738278097627"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:30:59.650969 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.650849 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2340311e-52f8-4b01-a04b-738278097627-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2340311e-52f8-4b01-a04b-738278097627" (UID: "2340311e-52f8-4b01-a04b-738278097627"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:30:59.650969 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.650881 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-kube-api-access-nhrhc" (OuterVolumeSpecName: "kube-api-access-nhrhc") pod "2340311e-52f8-4b01-a04b-738278097627" (UID: "2340311e-52f8-4b01-a04b-738278097627"). InnerVolumeSpecName "kube-api-access-nhrhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:30:59.750483 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.750446 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2340311e-52f8-4b01-a04b-738278097627-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:59.750483 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.750480 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhrhc\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-kube-api-access-nhrhc\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:59.750678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.750492 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2340311e-52f8-4b01-a04b-738278097627-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:59.750678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.750501 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2340311e-52f8-4b01-a04b-738278097627-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:30:59.750678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:30:59.750509 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2340311e-52f8-4b01-a04b-738278097627-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:00.140284 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.140251 2556 generic.go:356] "Generic (PLEG): container finished" podID="2340311e-52f8-4b01-a04b-738278097627" containerID="1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11" exitCode=0 Aug 04 16:31:00.140467 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.140300 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" Aug 04 16:31:00.140467 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.140332 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" event={"ID":"2340311e-52f8-4b01-a04b-738278097627","Type":"ContainerDied","Data":"1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11"} Aug 04 16:31:00.140467 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.140392 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f" event={"ID":"2340311e-52f8-4b01-a04b-738278097627","Type":"ContainerDied","Data":"b536e65af064c5d5ee5b127d76412598f95c674e7e182827ee6210e4cf220ec3"} Aug 04 16:31:00.140467 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.140410 2556 scope.go:117] "RemoveContainer" containerID="1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11" Aug 04 16:31:00.142055 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.142019 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" event={"ID":"dd0c7e98-91be-4ed4-8132-885c8ddd812f","Type":"ContainerStarted","Data":"10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05"} Aug 04 16:31:00.142178 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.142096 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:31:00.142178 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.142115 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" event={"ID":"dd0c7e98-91be-4ed4-8132-885c8ddd812f","Type":"ContainerStarted","Data":"839b757c99687947025d6795aa7af12251988e6b3aa2acbd21ab95fe212b28f3"} Aug 04 16:31:00.143754 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.143713 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" event={"ID":"61f3cdb7-8179-425a-b870-84b8ec00f293","Type":"ContainerStarted","Data":"d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a"} Aug 04 16:31:00.143864 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.143832 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:31:00.150609 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.150584 2556 scope.go:117] "RemoveContainer" containerID="7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3" Aug 04 16:31:00.159305 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.159282 2556 scope.go:117] "RemoveContainer" containerID="1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11" Aug 04 16:31:00.159557 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:31:00.159538 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11\": container with ID starting with 1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11 not found: ID does not exist" containerID="1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11" Aug 04 16:31:00.159621 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.159565 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11"} err="failed to get container status \"1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11\": rpc error: code = NotFound desc = could not find container \"1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11\": container with ID starting with 1d2bbbca0821813a974997fddc096e0e3cbe40611efef38030067af9e99c0c11 not found: ID does not exist" Aug 04 16:31:00.159621 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.159582 2556 scope.go:117] "RemoveContainer" containerID="7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3" Aug 04 16:31:00.159880 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:31:00.159863 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3\": container with ID starting with 7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3 not found: ID does not exist" containerID="7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3" Aug 04 16:31:00.159941 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.159884 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3"} err="failed to get container status \"7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3\": rpc error: code = NotFound desc = could not find container \"7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3\": container with ID starting with 7692a0bdfba46992ca89cfaa4f47d15aed5b40640e5147f824736b57b54df1a3 not found: ID does not exist" Aug 04 16:31:00.164587 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.164544 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" podStartSLOduration=2.164531436 podStartE2EDuration="2.164531436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:30:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:31:00.163606856 +0000 UTC m=+2089.486030336" watchObservedRunningTime="2026-08-04 16:31:00.164531436 +0000 UTC m=+2089.486954916" Aug 04 16:31:00.187228 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.187170 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" podStartSLOduration=2.187151683 podStartE2EDuration="2.187151683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:30:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:31:00.183906594 +0000 UTC m=+2089.506330076" watchObservedRunningTime="2026-08-04 16:31:00.187151683 +0000 UTC m=+2089.509575164" Aug 04 16:31:00.199454 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.199423 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f"] Aug 04 16:31:00.203655 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:00.203622 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3a345e2e-openshift-default-5f8cf545d4-2448f"] Aug 04 16:31:01.282150 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:01.282109 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2340311e-52f8-4b01-a04b-738278097627" path="/var/lib/kubelet/pods/2340311e-52f8-4b01-a04b-738278097627/volumes" Aug 04 16:31:11.152454 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:11.152417 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:31:11.152918 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:11.152609 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:31:11.210845 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:11.210806 2556 generic.go:356] "Generic (PLEG): container finished" podID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerID="914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2" exitCode=0 Aug 04 16:31:11.211064 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:11.210982 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" event={"ID":"052e0fa9-7e2e-4bb4-aa0c-d484c0338025","Type":"ContainerDied","Data":"914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2"} Aug 04 16:31:11.211530 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:11.211507 2556 scope.go:117] "RemoveContainer" containerID="914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2" Aug 04 16:31:11.213172 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:11.213130 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:31:11.213274 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:11.213232 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:31:11.213350 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:11.213285 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:31:12.120559 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:12.120522 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:31:12.217787 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:12.217750 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" event={"ID":"052e0fa9-7e2e-4bb4-aa0c-d484c0338025","Type":"ContainerStarted","Data":"018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2"} Aug 04 16:31:12.679689 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:12.679651 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:31:12.680084 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:12.680058 2556 patch_prober.go:28] interesting pod/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Aug 04 16:31:12.680255 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:12.680229 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Aug 04 16:31:13.678322 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:13.678271 2556 patch_prober.go:28] interesting pod/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Aug 04 16:31:13.678861 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:13.678348 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Aug 04 16:31:14.678902 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:14.678856 2556 patch_prober.go:28] interesting pod/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Aug 04 16:31:14.679420 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:14.678927 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Aug 04 16:31:15.678739 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:15.678670 2556 patch_prober.go:28] interesting pod/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Aug 04 16:31:15.678953 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:15.678763 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Aug 04 16:31:16.679129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:16.679078 2556 patch_prober.go:28] interesting pod/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Aug 04 16:31:16.679638 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:16.679152 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Aug 04 16:31:17.682120 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:17.682082 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:31:17.682597 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:17.682517 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:31:17.683235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:17.683215 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:31:29.233063 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.233030 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn"] Aug 04 16:31:29.233578 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.233276 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" podUID="dd0c7e98-91be-4ed4-8132-885c8ddd812f" containerName="payload-processing" containerID="cri-o://10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05" gracePeriod=30 Aug 04 16:31:29.241777 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.241716 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q"] Aug 04 16:31:29.241998 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.241959 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" podUID="61f3cdb7-8179-425a-b870-84b8ec00f293" containerName="payload-pre-processing" containerID="cri-o://d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a" gracePeriod=30 Aug 04 16:31:29.513603 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.513564 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:31:29.539899 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.538154 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995"] Aug 04 16:31:29.539899 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.538407 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" containerID="cri-o://018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2" gracePeriod=30 Aug 04 16:31:29.555211 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.555176 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:31:29.558066 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.558032 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0c7e98-91be-4ed4-8132-885c8ddd812f-plugins-config-volume\") pod \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\" (UID: \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\") " Aug 04 16:31:29.558221 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.558111 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktztc\" (UniqueName: \"kubernetes.io/projected/dd0c7e98-91be-4ed4-8132-885c8ddd812f-kube-api-access-ktztc\") pod \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\" (UID: \"dd0c7e98-91be-4ed4-8132-885c8ddd812f\") " Aug 04 16:31:29.559174 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.559124 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd0c7e98-91be-4ed4-8132-885c8ddd812f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd0c7e98-91be-4ed4-8132-885c8ddd812f" (UID: "dd0c7e98-91be-4ed4-8132-885c8ddd812f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:31:29.562663 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.562618 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd0c7e98-91be-4ed4-8132-885c8ddd812f-kube-api-access-ktztc" (OuterVolumeSpecName: "kube-api-access-ktztc") pod "dd0c7e98-91be-4ed4-8132-885c8ddd812f" (UID: "dd0c7e98-91be-4ed4-8132-885c8ddd812f"). InnerVolumeSpecName "kube-api-access-ktztc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:31:29.659632 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.659601 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61f3cdb7-8179-425a-b870-84b8ec00f293-plugins-config-volume\") pod \"61f3cdb7-8179-425a-b870-84b8ec00f293\" (UID: \"61f3cdb7-8179-425a-b870-84b8ec00f293\") " Aug 04 16:31:29.659831 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.659688 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d979x\" (UniqueName: \"kubernetes.io/projected/61f3cdb7-8179-425a-b870-84b8ec00f293-kube-api-access-d979x\") pod \"61f3cdb7-8179-425a-b870-84b8ec00f293\" (UID: \"61f3cdb7-8179-425a-b870-84b8ec00f293\") " Aug 04 16:31:29.659912 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.659898 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0c7e98-91be-4ed4-8132-885c8ddd812f-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:29.659951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.659914 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktztc\" (UniqueName: \"kubernetes.io/projected/dd0c7e98-91be-4ed4-8132-885c8ddd812f-kube-api-access-ktztc\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:29.660012 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.659989 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61f3cdb7-8179-425a-b870-84b8ec00f293-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "61f3cdb7-8179-425a-b870-84b8ec00f293" (UID: "61f3cdb7-8179-425a-b870-84b8ec00f293"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:31:29.661734 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.661700 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61f3cdb7-8179-425a-b870-84b8ec00f293-kube-api-access-d979x" (OuterVolumeSpecName: "kube-api-access-d979x") pod "61f3cdb7-8179-425a-b870-84b8ec00f293" (UID: "61f3cdb7-8179-425a-b870-84b8ec00f293"). InnerVolumeSpecName "kube-api-access-d979x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:31:29.760703 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.760611 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61f3cdb7-8179-425a-b870-84b8ec00f293-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:29.760703 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:29.760641 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d979x\" (UniqueName: \"kubernetes.io/projected/61f3cdb7-8179-425a-b870-84b8ec00f293-kube-api-access-d979x\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:30.318711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.318668 2556 generic.go:356] "Generic (PLEG): container finished" podID="dd0c7e98-91be-4ed4-8132-885c8ddd812f" containerID="10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05" exitCode=0 Aug 04 16:31:30.320158 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.319450 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" event={"ID":"dd0c7e98-91be-4ed4-8132-885c8ddd812f","Type":"ContainerDied","Data":"10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05"} Aug 04 16:31:30.320158 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.319527 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" event={"ID":"dd0c7e98-91be-4ed4-8132-885c8ddd812f","Type":"ContainerDied","Data":"839b757c99687947025d6795aa7af12251988e6b3aa2acbd21ab95fe212b28f3"} Aug 04 16:31:30.320158 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.319558 2556 scope.go:117] "RemoveContainer" containerID="10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05" Aug 04 16:31:30.320158 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.319462 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn" Aug 04 16:31:30.321129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.321097 2556 generic.go:356] "Generic (PLEG): container finished" podID="61f3cdb7-8179-425a-b870-84b8ec00f293" containerID="d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a" exitCode=0 Aug 04 16:31:30.321210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.321141 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" Aug 04 16:31:30.321210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.321156 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" event={"ID":"61f3cdb7-8179-425a-b870-84b8ec00f293","Type":"ContainerDied","Data":"d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a"} Aug 04 16:31:30.321210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.321202 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q" event={"ID":"61f3cdb7-8179-425a-b870-84b8ec00f293","Type":"ContainerDied","Data":"6164606f6368708afa0adea7ab5525a11bbb57d9ad17111a046b829ade7ab4b5"} Aug 04 16:31:30.329454 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.329361 2556 scope.go:117] "RemoveContainer" containerID="10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05" Aug 04 16:31:30.330274 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:31:30.330246 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05\": container with ID starting with 10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05 not found: ID does not exist" containerID="10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05" Aug 04 16:31:30.330394 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.330285 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05"} err="failed to get container status \"10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05\": rpc error: code = NotFound desc = could not find container \"10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05\": container with ID starting with 10d386d7f73edbabfa05d1b4a578cd09a0cb082e09a9248ba4381525dff92d05 not found: ID does not exist" Aug 04 16:31:30.330394 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.330308 2556 scope.go:117] "RemoveContainer" containerID="d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a" Aug 04 16:31:30.338804 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.338779 2556 scope.go:117] "RemoveContainer" containerID="d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a" Aug 04 16:31:30.339053 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:31:30.339035 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a\": container with ID starting with d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a not found: ID does not exist" containerID="d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a" Aug 04 16:31:30.339100 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.339059 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a"} err="failed to get container status \"d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a\": rpc error: code = NotFound desc = could not find container \"d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a\": container with ID starting with d6f5e811df604c844604e0d502dc5c8da66e2a220e91779c8ab9af2db9ac628a not found: ID does not exist" Aug 04 16:31:30.353325 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.353298 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q"] Aug 04 16:31:30.360341 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.360318 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-64639142-5bd756b896-6wp2q"] Aug 04 16:31:30.374367 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.374343 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn"] Aug 04 16:31:30.382123 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.382098 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-64639142-77f8976ffc-n68mn"] Aug 04 16:31:30.414775 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.414745 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7"] Aug 04 16:31:30.415147 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415134 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" Aug 04 16:31:30.415200 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415149 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" Aug 04 16:31:30.415200 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415158 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" Aug 04 16:31:30.415200 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415164 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" Aug 04 16:31:30.415200 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415182 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61f3cdb7-8179-425a-b870-84b8ec00f293" containerName="payload-pre-processing" Aug 04 16:31:30.415200 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415188 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="61f3cdb7-8179-425a-b870-84b8ec00f293" containerName="payload-pre-processing" Aug 04 16:31:30.415200 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415197 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd0c7e98-91be-4ed4-8132-885c8ddd812f" containerName="payload-processing" Aug 04 16:31:30.415200 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415203 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd0c7e98-91be-4ed4-8132-885c8ddd812f" containerName="payload-processing" Aug 04 16:31:30.415431 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415280 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd0c7e98-91be-4ed4-8132-885c8ddd812f" containerName="payload-processing" Aug 04 16:31:30.415431 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415290 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="61f3cdb7-8179-425a-b870-84b8ec00f293" containerName="payload-pre-processing" Aug 04 16:31:30.415431 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415296 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" Aug 04 16:31:30.415431 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.415303 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2340311e-52f8-4b01-a04b-738278097627" containerName="istio-proxy" Aug 04 16:31:30.419793 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.419773 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.422989 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.422970 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1978158b-openshift-default-dockercfg-zk4cx\"" Aug 04 16:31:30.436034 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.436002 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7"] Aug 04 16:31:30.467155 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.467104 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-credential-socket\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.467155 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.467149 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-data\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.467389 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.467230 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e965fb85-d997-4355-94b6-c33029e5021f-istio-podinfo\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.467389 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.467324 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-istio-token\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.467389 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.467377 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lttj\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-kube-api-access-8lttj\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.467563 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.467417 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-certs\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.467563 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.467466 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-envoy\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.467563 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.467504 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-socket\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.467734 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.467569 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e965fb85-d997-4355-94b6-c33029e5021f-istiod-ca-cert\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568185 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568084 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e965fb85-d997-4355-94b6-c33029e5021f-istiod-ca-cert\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568185 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568159 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-credential-socket\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568404 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568188 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-data\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568404 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568236 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e965fb85-d997-4355-94b6-c33029e5021f-istio-podinfo\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568404 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568280 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-istio-token\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568404 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568314 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lttj\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-kube-api-access-8lttj\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568404 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568342 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-certs\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568404 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568375 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-envoy\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568753 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568412 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-socket\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568830 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-socket\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568913 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568856 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-credential-socket\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.568913 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568889 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-data\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.569021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.568928 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-certs\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.569266 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.569249 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e965fb85-d997-4355-94b6-c33029e5021f-istiod-ca-cert\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.570719 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.570687 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-envoy\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.570866 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.570778 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e965fb85-d997-4355-94b6-c33029e5021f-istio-podinfo\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.577706 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.577674 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-istio-token\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.577890 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.577829 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lttj\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-kube-api-access-8lttj\") pod \"e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.730919 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.730879 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:30.880127 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.880100 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7"] Aug 04 16:31:30.880469 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:31:30.880444 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode965fb85_d997_4355_94b6_c33029e5021f.slice/crio-596572249aafa88ffd9b7acc90db989f77e8380f2102c5eb088be02d0d9d08e8 WatchSource:0}: Error finding container 596572249aafa88ffd9b7acc90db989f77e8380f2102c5eb088be02d0d9d08e8: Status 404 returned error can't find the container with id 596572249aafa88ffd9b7acc90db989f77e8380f2102c5eb088be02d0d9d08e8 Aug 04 16:31:30.882631 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.882600 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:31:30.882682 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.882667 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:31:30.882717 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:30.882696 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:31:31.273204 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:31.273171 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61f3cdb7-8179-425a-b870-84b8ec00f293" path="/var/lib/kubelet/pods/61f3cdb7-8179-425a-b870-84b8ec00f293/volumes" Aug 04 16:31:31.273577 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:31.273564 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd0c7e98-91be-4ed4-8132-885c8ddd812f" path="/var/lib/kubelet/pods/dd0c7e98-91be-4ed4-8132-885c8ddd812f/volumes" Aug 04 16:31:31.327630 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:31.327596 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" event={"ID":"e965fb85-d997-4355-94b6-c33029e5021f","Type":"ContainerStarted","Data":"bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16"} Aug 04 16:31:31.328117 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:31.327655 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" event={"ID":"e965fb85-d997-4355-94b6-c33029e5021f","Type":"ContainerStarted","Data":"596572249aafa88ffd9b7acc90db989f77e8380f2102c5eb088be02d0d9d08e8"} Aug 04 16:31:31.354251 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:31.354201 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" podStartSLOduration=1.354184698 podStartE2EDuration="1.354184698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:31:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:31:31.35132813 +0000 UTC m=+2120.673751622" watchObservedRunningTime="2026-08-04 16:31:31.354184698 +0000 UTC m=+2120.676608183" Aug 04 16:31:31.731459 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:31.731366 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:31.736620 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:31.736586 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:32.335790 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:32.335750 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:32.336821 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:32.336793 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:32.685781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:32.685681 2556 patch_prober.go:28] interesting pod/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 16:31:32.685926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:32.685778 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:31:34.484982 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.484942 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx"] Aug 04 16:31:34.489738 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.489705 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:34.492893 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.492865 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1978158b-dockercfg-nnst5\"" Aug 04 16:31:34.493024 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.492889 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1978158b\"" Aug 04 16:31:34.499639 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.499614 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx"] Aug 04 16:31:34.518052 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.518023 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc"] Aug 04 16:31:34.521644 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.521625 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:34.529778 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.529752 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc"] Aug 04 16:31:34.617085 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.617051 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45bjq\" (UniqueName: \"kubernetes.io/projected/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-kube-api-access-45bjq\") pod \"payload-processing-e2e-ait-1978158b-794999b946-bxbqc\" (UID: \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\") " pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:34.617259 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.617117 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b1be34-19ec-43c8-950d-9f07c102c8bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx\" (UID: \"69b1be34-19ec-43c8-950d-9f07c102c8bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:34.617259 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.617139 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-plugins-config-volume\") pod \"payload-processing-e2e-ait-1978158b-794999b946-bxbqc\" (UID: \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\") " pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:34.617259 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.617195 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqh2b\" (UniqueName: \"kubernetes.io/projected/69b1be34-19ec-43c8-950d-9f07c102c8bf-kube-api-access-hqh2b\") pod \"payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx\" (UID: \"69b1be34-19ec-43c8-950d-9f07c102c8bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:34.718435 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.718397 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b1be34-19ec-43c8-950d-9f07c102c8bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx\" (UID: \"69b1be34-19ec-43c8-950d-9f07c102c8bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:34.718435 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.718446 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-plugins-config-volume\") pod \"payload-processing-e2e-ait-1978158b-794999b946-bxbqc\" (UID: \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\") " pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:34.718662 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.718516 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqh2b\" (UniqueName: \"kubernetes.io/projected/69b1be34-19ec-43c8-950d-9f07c102c8bf-kube-api-access-hqh2b\") pod \"payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx\" (UID: \"69b1be34-19ec-43c8-950d-9f07c102c8bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:34.718662 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.718604 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45bjq\" (UniqueName: \"kubernetes.io/projected/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-kube-api-access-45bjq\") pod \"payload-processing-e2e-ait-1978158b-794999b946-bxbqc\" (UID: \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\") " pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:34.719688 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.719662 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-plugins-config-volume\") pod \"payload-processing-e2e-ait-1978158b-794999b946-bxbqc\" (UID: \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\") " pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:34.719985 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.719961 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b1be34-19ec-43c8-950d-9f07c102c8bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx\" (UID: \"69b1be34-19ec-43c8-950d-9f07c102c8bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:34.727172 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.727141 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45bjq\" (UniqueName: \"kubernetes.io/projected/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-kube-api-access-45bjq\") pod \"payload-processing-e2e-ait-1978158b-794999b946-bxbqc\" (UID: \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\") " pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:34.727476 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.727457 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqh2b\" (UniqueName: \"kubernetes.io/projected/69b1be34-19ec-43c8-950d-9f07c102c8bf-kube-api-access-hqh2b\") pod \"payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx\" (UID: \"69b1be34-19ec-43c8-950d-9f07c102c8bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:34.801868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.801836 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:34.815940 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.815918 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:31:34.833382 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.833357 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:34.920321 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.920277 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-envoy\") pod \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " Aug 04 16:31:34.920500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.920333 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-certs\") pod \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " Aug 04 16:31:34.920500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.920377 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttmlj\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-kube-api-access-ttmlj\") pod \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " Aug 04 16:31:34.920500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.920406 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-data\") pod \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " Aug 04 16:31:34.920500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.920470 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-podinfo\") pod \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " Aug 04 16:31:34.920709 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.920504 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-socket\") pod \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " Aug 04 16:31:34.920709 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.920561 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-token\") pod \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " Aug 04 16:31:34.920709 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.920588 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istiod-ca-cert\") pod \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " Aug 04 16:31:34.920709 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.920640 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-credential-socket\") pod \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\" (UID: \"052e0fa9-7e2e-4bb4-aa0c-d484c0338025\") " Aug 04 16:31:34.921693 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.921214 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-data" (OuterVolumeSpecName: "istio-data") pod "052e0fa9-7e2e-4bb4-aa0c-d484c0338025" (UID: "052e0fa9-7e2e-4bb4-aa0c-d484c0338025"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:31:34.921693 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.921281 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "052e0fa9-7e2e-4bb4-aa0c-d484c0338025" (UID: "052e0fa9-7e2e-4bb4-aa0c-d484c0338025"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:31:34.921963 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.921758 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "052e0fa9-7e2e-4bb4-aa0c-d484c0338025" (UID: "052e0fa9-7e2e-4bb4-aa0c-d484c0338025"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:31:34.921963 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.921889 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "052e0fa9-7e2e-4bb4-aa0c-d484c0338025" (UID: "052e0fa9-7e2e-4bb4-aa0c-d484c0338025"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:31:34.922066 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.921987 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "052e0fa9-7e2e-4bb4-aa0c-d484c0338025" (UID: "052e0fa9-7e2e-4bb4-aa0c-d484c0338025"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:31:34.924209 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.924183 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "052e0fa9-7e2e-4bb4-aa0c-d484c0338025" (UID: "052e0fa9-7e2e-4bb4-aa0c-d484c0338025"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:31:34.925770 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.925747 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "052e0fa9-7e2e-4bb4-aa0c-d484c0338025" (UID: "052e0fa9-7e2e-4bb4-aa0c-d484c0338025"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:31:34.926401 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.926368 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-token" (OuterVolumeSpecName: "istio-token") pod "052e0fa9-7e2e-4bb4-aa0c-d484c0338025" (UID: "052e0fa9-7e2e-4bb4-aa0c-d484c0338025"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:31:34.927008 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.926974 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-kube-api-access-ttmlj" (OuterVolumeSpecName: "kube-api-access-ttmlj") pod "052e0fa9-7e2e-4bb4-aa0c-d484c0338025" (UID: "052e0fa9-7e2e-4bb4-aa0c-d484c0338025"). InnerVolumeSpecName "kube-api-access-ttmlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:31:34.975116 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:34.975032 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx"] Aug 04 16:31:34.980101 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:31:34.980071 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b1be34_19ec_43c8_950d_9f07c102c8bf.slice/crio-2b544b917fe4bb32bf59c359778a73570474118411e0d4aa181887116991114b WatchSource:0}: Error finding container 2b544b917fe4bb32bf59c359778a73570474118411e0d4aa181887116991114b: Status 404 returned error can't find the container with id 2b544b917fe4bb32bf59c359778a73570474118411e0d4aa181887116991114b Aug 04 16:31:35.005645 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.005618 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc"] Aug 04 16:31:35.007888 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:31:35.007864 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode67aec1e_b1c3_4a2b_a23d_bfeffbf01664.slice/crio-5fb9fd7ad57a74a7fa4b8a10260d5853ec670e59de7c366e246300bf88b2daef WatchSource:0}: Error finding container 5fb9fd7ad57a74a7fa4b8a10260d5853ec670e59de7c366e246300bf88b2daef: Status 404 returned error can't find the container with id 5fb9fd7ad57a74a7fa4b8a10260d5853ec670e59de7c366e246300bf88b2daef Aug 04 16:31:35.021984 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.021963 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:35.021984 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.021984 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:35.022146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.021993 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:35.022146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.022004 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:35.022146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.022012 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:35.022146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.022020 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:35.022146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.022027 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:35.022146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.022035 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:35.022146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.022043 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttmlj\" (UniqueName: \"kubernetes.io/projected/052e0fa9-7e2e-4bb4-aa0c-d484c0338025-kube-api-access-ttmlj\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:31:35.355835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.355748 2556 generic.go:356] "Generic (PLEG): container finished" podID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerID="018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2" exitCode=0 Aug 04 16:31:35.355835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.355799 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" Aug 04 16:31:35.355835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.355819 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" event={"ID":"052e0fa9-7e2e-4bb4-aa0c-d484c0338025","Type":"ContainerDied","Data":"018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2"} Aug 04 16:31:35.356101 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.355871 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995" event={"ID":"052e0fa9-7e2e-4bb4-aa0c-d484c0338025","Type":"ContainerDied","Data":"1703774a6692525f6a7159ef26b61d96421dfbf6d81d8ae25396c0db793e8f2d"} Aug 04 16:31:35.356101 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.355899 2556 scope.go:117] "RemoveContainer" containerID="018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2" Aug 04 16:31:35.357563 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.357521 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" event={"ID":"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664","Type":"ContainerStarted","Data":"f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f"} Aug 04 16:31:35.357687 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.357576 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" event={"ID":"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664","Type":"ContainerStarted","Data":"5fb9fd7ad57a74a7fa4b8a10260d5853ec670e59de7c366e246300bf88b2daef"} Aug 04 16:31:35.357687 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.357604 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:35.359433 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.359408 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" event={"ID":"69b1be34-19ec-43c8-950d-9f07c102c8bf","Type":"ContainerStarted","Data":"e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d"} Aug 04 16:31:35.359538 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.359450 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" event={"ID":"69b1be34-19ec-43c8-950d-9f07c102c8bf","Type":"ContainerStarted","Data":"2b544b917fe4bb32bf59c359778a73570474118411e0d4aa181887116991114b"} Aug 04 16:31:35.359538 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.359480 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:35.369487 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.369460 2556 scope.go:117] "RemoveContainer" containerID="914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2" Aug 04 16:31:35.379063 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.379039 2556 scope.go:117] "RemoveContainer" containerID="018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2" Aug 04 16:31:35.379380 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:31:35.379357 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2\": container with ID starting with 018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2 not found: ID does not exist" containerID="018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2" Aug 04 16:31:35.379449 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.379388 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2"} err="failed to get container status \"018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2\": rpc error: code = NotFound desc = could not find container \"018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2\": container with ID starting with 018ecaa88f757450e26510b4e6f7570506d51de5ea5b45bb52c1499e8ba82ac2 not found: ID does not exist" Aug 04 16:31:35.379449 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.379408 2556 scope.go:117] "RemoveContainer" containerID="914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2" Aug 04 16:31:35.379657 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:31:35.379635 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2\": container with ID starting with 914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2 not found: ID does not exist" containerID="914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2" Aug 04 16:31:35.379712 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.379665 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2"} err="failed to get container status \"914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2\": rpc error: code = NotFound desc = could not find container \"914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2\": container with ID starting with 914087875d910562afd0fa14db17def4913e743423bd324467e64263008af5b2 not found: ID does not exist" Aug 04 16:31:35.384387 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.384331 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" podStartSLOduration=1.3843187559999999 podStartE2EDuration="1.384318756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:31:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:31:35.381957695 +0000 UTC m=+2124.704381183" watchObservedRunningTime="2026-08-04 16:31:35.384318756 +0000 UTC m=+2124.706742271" Aug 04 16:31:35.402044 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.401993 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" podStartSLOduration=1.401975406 podStartE2EDuration="1.401975406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:31:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:31:35.401777271 +0000 UTC m=+2124.724200753" watchObservedRunningTime="2026-08-04 16:31:35.401975406 +0000 UTC m=+2124.724398886" Aug 04 16:31:35.417903 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.417875 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995"] Aug 04 16:31:35.422021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:35.421997 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-64639142-gw-openshift-default-8565d4f984-nv995"] Aug 04 16:31:37.273900 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:37.273870 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" path="/var/lib/kubelet/pods/052e0fa9-7e2e-4bb4-aa0c-d484c0338025/volumes" Aug 04 16:31:46.369001 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:46.368968 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:31:46.369421 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:46.369053 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:31:46.431179 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:46.431148 2556 generic.go:356] "Generic (PLEG): container finished" podID="e965fb85-d997-4355-94b6-c33029e5021f" containerID="bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16" exitCode=0 Aug 04 16:31:46.431340 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:46.431226 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" event={"ID":"e965fb85-d997-4355-94b6-c33029e5021f","Type":"ContainerDied","Data":"bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16"} Aug 04 16:31:46.431735 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:46.431712 2556 scope.go:117] "RemoveContainer" containerID="bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16" Aug 04 16:31:46.432859 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:46.432830 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:31:46.432984 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:46.432896 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:31:46.432984 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:46.432933 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:31:47.336182 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:47.336142 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:47.438607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:47.438569 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" event={"ID":"e965fb85-d997-4355-94b6-c33029e5021f","Type":"ContainerStarted","Data":"7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309"} Aug 04 16:31:47.731889 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:47.731789 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:47.733015 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:47.732986 2556 patch_prober.go:28] interesting pod/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Aug 04 16:31:47.733142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:47.733040 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Aug 04 16:31:48.731712 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:48.731673 2556 patch_prober.go:28] interesting pod/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Aug 04 16:31:48.732143 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:48.731763 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Aug 04 16:31:49.732336 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:49.732294 2556 patch_prober.go:28] interesting pod/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Aug 04 16:31:49.732895 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:49.732378 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Aug 04 16:31:50.731855 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:50.731815 2556 patch_prober.go:28] interesting pod/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Aug 04 16:31:50.732069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:50.731876 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Aug 04 16:31:51.731781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:51.731743 2556 patch_prober.go:28] interesting pod/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Aug 04 16:31:51.732180 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:51.731795 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Aug 04 16:31:52.736668 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:52.736636 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:52.737144 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:52.737003 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:31:52.737800 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:31:52.737784 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:32:04.581271 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.581184 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc"] Aug 04 16:32:04.581963 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.581411 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" podUID="e67aec1e-b1c3-4a2b-a23d-bfeffbf01664" containerName="payload-processing" containerID="cri-o://f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f" gracePeriod=30 Aug 04 16:32:04.587212 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.587185 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx"] Aug 04 16:32:04.587448 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.587422 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" podUID="69b1be34-19ec-43c8-950d-9f07c102c8bf" containerName="payload-pre-processing" containerID="cri-o://e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d" gracePeriod=30 Aug 04 16:32:04.882354 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.882315 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:32:04.887160 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.887130 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:32:04.912306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.912275 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-plugins-config-volume\") pod \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\" (UID: \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\") " Aug 04 16:32:04.912509 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.912342 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45bjq\" (UniqueName: \"kubernetes.io/projected/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-kube-api-access-45bjq\") pod \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\" (UID: \"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664\") " Aug 04 16:32:04.912509 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.912408 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqh2b\" (UniqueName: \"kubernetes.io/projected/69b1be34-19ec-43c8-950d-9f07c102c8bf-kube-api-access-hqh2b\") pod \"69b1be34-19ec-43c8-950d-9f07c102c8bf\" (UID: \"69b1be34-19ec-43c8-950d-9f07c102c8bf\") " Aug 04 16:32:04.912509 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.912487 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b1be34-19ec-43c8-950d-9f07c102c8bf-plugins-config-volume\") pod \"69b1be34-19ec-43c8-950d-9f07c102c8bf\" (UID: \"69b1be34-19ec-43c8-950d-9f07c102c8bf\") " Aug 04 16:32:04.912810 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.912676 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e67aec1e-b1c3-4a2b-a23d-bfeffbf01664" (UID: "e67aec1e-b1c3-4a2b-a23d-bfeffbf01664"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:32:04.912881 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.912856 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:04.913097 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.913063 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69b1be34-19ec-43c8-950d-9f07c102c8bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69b1be34-19ec-43c8-950d-9f07c102c8bf" (UID: "69b1be34-19ec-43c8-950d-9f07c102c8bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:32:04.914844 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.914813 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69b1be34-19ec-43c8-950d-9f07c102c8bf-kube-api-access-hqh2b" (OuterVolumeSpecName: "kube-api-access-hqh2b") pod "69b1be34-19ec-43c8-950d-9f07c102c8bf" (UID: "69b1be34-19ec-43c8-950d-9f07c102c8bf"). InnerVolumeSpecName "kube-api-access-hqh2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:32:04.914954 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:04.914841 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-kube-api-access-45bjq" (OuterVolumeSpecName: "kube-api-access-45bjq") pod "e67aec1e-b1c3-4a2b-a23d-bfeffbf01664" (UID: "e67aec1e-b1c3-4a2b-a23d-bfeffbf01664"). InnerVolumeSpecName "kube-api-access-45bjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:32:05.014235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.014200 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45bjq\" (UniqueName: \"kubernetes.io/projected/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664-kube-api-access-45bjq\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:05.014235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.014230 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqh2b\" (UniqueName: \"kubernetes.io/projected/69b1be34-19ec-43c8-950d-9f07c102c8bf-kube-api-access-hqh2b\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:05.014235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.014239 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b1be34-19ec-43c8-950d-9f07c102c8bf-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:05.535920 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.535831 2556 generic.go:356] "Generic (PLEG): container finished" podID="e67aec1e-b1c3-4a2b-a23d-bfeffbf01664" containerID="f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f" exitCode=0 Aug 04 16:32:05.535920 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.535870 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" Aug 04 16:32:05.536161 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.535921 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" event={"ID":"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664","Type":"ContainerDied","Data":"f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f"} Aug 04 16:32:05.536161 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.535976 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc" event={"ID":"e67aec1e-b1c3-4a2b-a23d-bfeffbf01664","Type":"ContainerDied","Data":"5fb9fd7ad57a74a7fa4b8a10260d5853ec670e59de7c366e246300bf88b2daef"} Aug 04 16:32:05.536161 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.536008 2556 scope.go:117] "RemoveContainer" containerID="f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f" Aug 04 16:32:05.537499 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.537472 2556 generic.go:356] "Generic (PLEG): container finished" podID="69b1be34-19ec-43c8-950d-9f07c102c8bf" containerID="e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d" exitCode=0 Aug 04 16:32:05.537603 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.537515 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" Aug 04 16:32:05.537603 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.537558 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" event={"ID":"69b1be34-19ec-43c8-950d-9f07c102c8bf","Type":"ContainerDied","Data":"e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d"} Aug 04 16:32:05.537692 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.537610 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx" event={"ID":"69b1be34-19ec-43c8-950d-9f07c102c8bf","Type":"ContainerDied","Data":"2b544b917fe4bb32bf59c359778a73570474118411e0d4aa181887116991114b"} Aug 04 16:32:05.544998 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.544974 2556 scope.go:117] "RemoveContainer" containerID="f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f" Aug 04 16:32:05.545269 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:32:05.545245 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f\": container with ID starting with f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f not found: ID does not exist" containerID="f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f" Aug 04 16:32:05.545325 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.545281 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f"} err="failed to get container status \"f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f\": rpc error: code = NotFound desc = could not find container \"f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f\": container with ID starting with f3b8321efb2bf493fd322fcdb87f399eba476fa5e2dde9c97aaf5bdc48cf873f not found: ID does not exist" Aug 04 16:32:05.545325 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.545298 2556 scope.go:117] "RemoveContainer" containerID="e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d" Aug 04 16:32:05.553441 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.553369 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx"] Aug 04 16:32:05.553502 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.553458 2556 scope.go:117] "RemoveContainer" containerID="e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d" Aug 04 16:32:05.553756 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:32:05.553735 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d\": container with ID starting with e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d not found: ID does not exist" containerID="e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d" Aug 04 16:32:05.553851 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.553761 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d"} err="failed to get container status \"e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d\": rpc error: code = NotFound desc = could not find container \"e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d\": container with ID starting with e32fd75cff718d4b4f6f92a1158afe7b43efac0c43d24f12b357046cb52b0b8d not found: ID does not exist" Aug 04 16:32:05.558499 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.558479 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1978158b-79479485c6-wj9tx"] Aug 04 16:32:05.572188 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.572161 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc"] Aug 04 16:32:05.576103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.576083 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1978158b-794999b946-bxbqc"] Aug 04 16:32:05.967593 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.967500 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7"] Aug 04 16:32:05.968115 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:05.968041 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" containerID="cri-o://7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309" gracePeriod=30 Aug 04 16:32:06.751057 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751021 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk"] Aug 04 16:32:06.751515 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751497 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" Aug 04 16:32:06.751613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751519 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" Aug 04 16:32:06.751613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751541 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67aec1e-b1c3-4a2b-a23d-bfeffbf01664" containerName="payload-processing" Aug 04 16:32:06.751613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751550 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67aec1e-b1c3-4a2b-a23d-bfeffbf01664" containerName="payload-processing" Aug 04 16:32:06.751613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751594 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69b1be34-19ec-43c8-950d-9f07c102c8bf" containerName="payload-pre-processing" Aug 04 16:32:06.751613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751610 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="69b1be34-19ec-43c8-950d-9f07c102c8bf" containerName="payload-pre-processing" Aug 04 16:32:06.751893 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751740 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="69b1be34-19ec-43c8-950d-9f07c102c8bf" containerName="payload-pre-processing" Aug 04 16:32:06.751893 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751753 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" Aug 04 16:32:06.751893 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.751769 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67aec1e-b1c3-4a2b-a23d-bfeffbf01664" containerName="payload-processing" Aug 04 16:32:06.756287 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.756263 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.758975 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.758955 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-0eb193d3-openshift-default-dockercfg-92xxj\"" Aug 04 16:32:06.767111 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.767087 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk"] Aug 04 16:32:06.831669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.831390 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-istio-token\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.831669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.831459 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b383e43e-5c80-40b8-883c-280776d08f20-istio-podinfo\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.831669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.831499 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-certs\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.831669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.831541 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-socket\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.831669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.831586 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b383e43e-5c80-40b8-883c-280776d08f20-istiod-ca-cert\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.831669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.831618 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-data\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.831669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.831663 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rbdz\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-kube-api-access-6rbdz\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.831669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.831703 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-envoy\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.832218 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.831789 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-credential-socket\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.932576 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.932527 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-credential-socket\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.932840 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.932610 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-istio-token\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.932840 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.932666 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b383e43e-5c80-40b8-883c-280776d08f20-istio-podinfo\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.932840 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.932700 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-certs\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.932840 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.932755 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-socket\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.933072 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.932903 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b383e43e-5c80-40b8-883c-280776d08f20-istiod-ca-cert\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.933072 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.932961 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-data\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.933072 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.933025 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-credential-socket\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.933072 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.933014 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rbdz\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-kube-api-access-6rbdz\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.933281 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.933096 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-envoy\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.933281 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.933143 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-certs\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.933281 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.933192 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-socket\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.933496 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.933430 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-data\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.933773 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.933750 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b383e43e-5c80-40b8-883c-280776d08f20-istiod-ca-cert\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.935454 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.935433 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b383e43e-5c80-40b8-883c-280776d08f20-istio-podinfo\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.935580 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.935564 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-envoy\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.953718 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.950011 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-istio-token\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:06.954536 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:06.954504 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rbdz\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-kube-api-access-6rbdz\") pod \"e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:07.068494 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.068403 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:07.227773 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.227584 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk"] Aug 04 16:32:07.230303 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:32:07.230281 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb383e43e_5c80_40b8_883c_280776d08f20.slice/crio-72ea5aedaafd51da8eb1a1071eed9b2d409ec2b76563d4d923dea03114283384 WatchSource:0}: Error finding container 72ea5aedaafd51da8eb1a1071eed9b2d409ec2b76563d4d923dea03114283384: Status 404 returned error can't find the container with id 72ea5aedaafd51da8eb1a1071eed9b2d409ec2b76563d4d923dea03114283384 Aug 04 16:32:07.232442 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.232408 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:32:07.232527 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.232488 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:32:07.232593 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.232533 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:32:07.276532 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.276501 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69b1be34-19ec-43c8-950d-9f07c102c8bf" path="/var/lib/kubelet/pods/69b1be34-19ec-43c8-950d-9f07c102c8bf/volumes" Aug 04 16:32:07.277021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.277003 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e67aec1e-b1c3-4a2b-a23d-bfeffbf01664" path="/var/lib/kubelet/pods/e67aec1e-b1c3-4a2b-a23d-bfeffbf01664/volumes" Aug 04 16:32:07.550878 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.550843 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" event={"ID":"b383e43e-5c80-40b8-883c-280776d08f20","Type":"ContainerStarted","Data":"6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206"} Aug 04 16:32:07.551054 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.550896 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" event={"ID":"b383e43e-5c80-40b8-883c-280776d08f20","Type":"ContainerStarted","Data":"72ea5aedaafd51da8eb1a1071eed9b2d409ec2b76563d4d923dea03114283384"} Aug 04 16:32:07.573696 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.573638 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" podStartSLOduration=1.573619537 podStartE2EDuration="1.573619537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:32:07.573270481 +0000 UTC m=+2156.895693963" watchObservedRunningTime="2026-08-04 16:32:07.573619537 +0000 UTC m=+2156.896043017" Aug 04 16:32:07.739071 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.739025 2556 patch_prober.go:28] interesting pod/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 16:32:07.739235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:07.739113 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:32:08.069381 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:08.069346 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:08.074532 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:08.074504 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:08.556331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:08.556304 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:08.557320 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:08.557295 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:10.812154 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.812117 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6"] Aug 04 16:32:10.812551 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.812539 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" Aug 04 16:32:10.812592 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.812552 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" Aug 04 16:32:10.812637 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.812627 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="052e0fa9-7e2e-4bb4-aa0c-d484c0338025" containerName="istio-proxy" Aug 04 16:32:10.815989 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.815964 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:10.819060 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.819035 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-0eb193d3-dockercfg-9mhzh\"" Aug 04 16:32:10.820521 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.820501 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-0eb193d3\"" Aug 04 16:32:10.826330 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.826305 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6"] Aug 04 16:32:10.862169 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.862137 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c"] Aug 04 16:32:10.865715 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.865695 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:10.875002 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.874969 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/381eb171-64e1-4f69-8420-a4243f2f91fd-plugins-config-volume\") pod \"payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6\" (UID: \"381eb171-64e1-4f69-8420-a4243f2f91fd\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:10.875179 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.875157 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4kmz\" (UniqueName: \"kubernetes.io/projected/381eb171-64e1-4f69-8420-a4243f2f91fd-kube-api-access-q4kmz\") pod \"payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6\" (UID: \"381eb171-64e1-4f69-8420-a4243f2f91fd\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:10.875685 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.875667 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c"] Aug 04 16:32:10.976276 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.976238 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26640232-107e-447e-af20-d65af4be88f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c\" (UID: \"26640232-107e-447e-af20-d65af4be88f4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:10.976426 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.976311 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/381eb171-64e1-4f69-8420-a4243f2f91fd-plugins-config-volume\") pod \"payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6\" (UID: \"381eb171-64e1-4f69-8420-a4243f2f91fd\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:10.976426 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.976362 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpkgf\" (UniqueName: \"kubernetes.io/projected/26640232-107e-447e-af20-d65af4be88f4-kube-api-access-zpkgf\") pod \"payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c\" (UID: \"26640232-107e-447e-af20-d65af4be88f4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:10.976534 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.976442 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4kmz\" (UniqueName: \"kubernetes.io/projected/381eb171-64e1-4f69-8420-a4243f2f91fd-kube-api-access-q4kmz\") pod \"payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6\" (UID: \"381eb171-64e1-4f69-8420-a4243f2f91fd\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:10.977177 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.977154 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/381eb171-64e1-4f69-8420-a4243f2f91fd-plugins-config-volume\") pod \"payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6\" (UID: \"381eb171-64e1-4f69-8420-a4243f2f91fd\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:10.985157 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:10.985134 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4kmz\" (UniqueName: \"kubernetes.io/projected/381eb171-64e1-4f69-8420-a4243f2f91fd-kube-api-access-q4kmz\") pod \"payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6\" (UID: \"381eb171-64e1-4f69-8420-a4243f2f91fd\") " pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:11.078142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.078056 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26640232-107e-447e-af20-d65af4be88f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c\" (UID: \"26640232-107e-447e-af20-d65af4be88f4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:11.078142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.078124 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpkgf\" (UniqueName: \"kubernetes.io/projected/26640232-107e-447e-af20-d65af4be88f4-kube-api-access-zpkgf\") pod \"payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c\" (UID: \"26640232-107e-447e-af20-d65af4be88f4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:11.078827 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.078802 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26640232-107e-447e-af20-d65af4be88f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c\" (UID: \"26640232-107e-447e-af20-d65af4be88f4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:11.087248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.087228 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpkgf\" (UniqueName: \"kubernetes.io/projected/26640232-107e-447e-af20-d65af4be88f4-kube-api-access-zpkgf\") pod \"payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c\" (UID: \"26640232-107e-447e-af20-d65af4be88f4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:11.131345 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.131306 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-0eb193d3-dockercfg-9mhzh\"" Aug 04 16:32:11.139075 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.139040 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:11.176874 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.176817 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:11.229352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.228918 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:32:11.280025 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.279990 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-certs\") pod \"e965fb85-d997-4355-94b6-c33029e5021f\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " Aug 04 16:32:11.280211 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280052 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e965fb85-d997-4355-94b6-c33029e5021f-istio-podinfo\") pod \"e965fb85-d997-4355-94b6-c33029e5021f\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " Aug 04 16:32:11.280211 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280119 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-data\") pod \"e965fb85-d997-4355-94b6-c33029e5021f\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " Aug 04 16:32:11.280211 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280159 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-istio-token\") pod \"e965fb85-d997-4355-94b6-c33029e5021f\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " Aug 04 16:32:11.280211 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280209 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-socket\") pod \"e965fb85-d997-4355-94b6-c33029e5021f\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " Aug 04 16:32:11.280432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280243 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e965fb85-d997-4355-94b6-c33029e5021f-istiod-ca-cert\") pod \"e965fb85-d997-4355-94b6-c33029e5021f\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " Aug 04 16:32:11.280432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280285 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-envoy\") pod \"e965fb85-d997-4355-94b6-c33029e5021f\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " Aug 04 16:32:11.280432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280325 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-credential-socket\") pod \"e965fb85-d997-4355-94b6-c33029e5021f\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " Aug 04 16:32:11.280432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280360 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lttj\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-kube-api-access-8lttj\") pod \"e965fb85-d997-4355-94b6-c33029e5021f\" (UID: \"e965fb85-d997-4355-94b6-c33029e5021f\") " Aug 04 16:32:11.280432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280352 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e965fb85-d997-4355-94b6-c33029e5021f" (UID: "e965fb85-d997-4355-94b6-c33029e5021f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:11.280432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280423 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e965fb85-d997-4355-94b6-c33029e5021f" (UID: "e965fb85-d997-4355-94b6-c33029e5021f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:11.280774 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280670 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:11.280774 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280688 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:11.280898 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.280772 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e965fb85-d997-4355-94b6-c33029e5021f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e965fb85-d997-4355-94b6-c33029e5021f" (UID: "e965fb85-d997-4355-94b6-c33029e5021f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:32:11.281067 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.281044 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e965fb85-d997-4355-94b6-c33029e5021f" (UID: "e965fb85-d997-4355-94b6-c33029e5021f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:11.281347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.281322 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e965fb85-d997-4355-94b6-c33029e5021f" (UID: "e965fb85-d997-4355-94b6-c33029e5021f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:11.283926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.283884 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e965fb85-d997-4355-94b6-c33029e5021f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e965fb85-d997-4355-94b6-c33029e5021f" (UID: "e965fb85-d997-4355-94b6-c33029e5021f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:32:11.283926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.283913 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e965fb85-d997-4355-94b6-c33029e5021f" (UID: "e965fb85-d997-4355-94b6-c33029e5021f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:32:11.284060 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.284020 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e965fb85-d997-4355-94b6-c33029e5021f" (UID: "e965fb85-d997-4355-94b6-c33029e5021f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:11.284060 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.284043 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-kube-api-access-8lttj" (OuterVolumeSpecName: "kube-api-access-8lttj") pod "e965fb85-d997-4355-94b6-c33029e5021f" (UID: "e965fb85-d997-4355-94b6-c33029e5021f"). InnerVolumeSpecName "kube-api-access-8lttj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:32:11.325905 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.325870 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6"] Aug 04 16:32:11.327071 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:32:11.327043 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381eb171_64e1_4f69_8420_a4243f2f91fd.slice/crio-b6276d64e8b01dbb033969d34829cec1a4dbf9a371cf48dc2979fe735c395d1a WatchSource:0}: Error finding container b6276d64e8b01dbb033969d34829cec1a4dbf9a371cf48dc2979fe735c395d1a: Status 404 returned error can't find the container with id b6276d64e8b01dbb033969d34829cec1a4dbf9a371cf48dc2979fe735c395d1a Aug 04 16:32:11.348094 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.348068 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c"] Aug 04 16:32:11.349944 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:32:11.349918 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26640232_107e_447e_af20_d65af4be88f4.slice/crio-b7fa048eb682867928f240b4a083c1c7a42faeed93eeaf6113c28d6ab7f92340 WatchSource:0}: Error finding container b7fa048eb682867928f240b4a083c1c7a42faeed93eeaf6113c28d6ab7f92340: Status 404 returned error can't find the container with id b7fa048eb682867928f240b4a083c1c7a42faeed93eeaf6113c28d6ab7f92340 Aug 04 16:32:11.381500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.381467 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e965fb85-d997-4355-94b6-c33029e5021f-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:11.381500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.381497 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:11.381720 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.381510 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:11.381720 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.381524 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lttj\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-kube-api-access-8lttj\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:11.381720 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.381540 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e965fb85-d997-4355-94b6-c33029e5021f-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:11.381720 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.381553 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e965fb85-d997-4355-94b6-c33029e5021f-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:11.381720 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.381569 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e965fb85-d997-4355-94b6-c33029e5021f-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:11.574495 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.574460 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" event={"ID":"26640232-107e-447e-af20-d65af4be88f4","Type":"ContainerStarted","Data":"03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e"} Aug 04 16:32:11.574701 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.574511 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" event={"ID":"26640232-107e-447e-af20-d65af4be88f4","Type":"ContainerStarted","Data":"b7fa048eb682867928f240b4a083c1c7a42faeed93eeaf6113c28d6ab7f92340"} Aug 04 16:32:11.574701 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.574642 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:11.576252 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.576224 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" event={"ID":"381eb171-64e1-4f69-8420-a4243f2f91fd","Type":"ContainerStarted","Data":"e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df"} Aug 04 16:32:11.576391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.576281 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:11.576391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.576306 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" event={"ID":"381eb171-64e1-4f69-8420-a4243f2f91fd","Type":"ContainerStarted","Data":"b6276d64e8b01dbb033969d34829cec1a4dbf9a371cf48dc2979fe735c395d1a"} Aug 04 16:32:11.578447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.578172 2556 generic.go:356] "Generic (PLEG): container finished" podID="e965fb85-d997-4355-94b6-c33029e5021f" containerID="7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309" exitCode=0 Aug 04 16:32:11.578447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.578208 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" event={"ID":"e965fb85-d997-4355-94b6-c33029e5021f","Type":"ContainerDied","Data":"7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309"} Aug 04 16:32:11.578447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.578227 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" Aug 04 16:32:11.578447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.578283 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7" event={"ID":"e965fb85-d997-4355-94b6-c33029e5021f","Type":"ContainerDied","Data":"596572249aafa88ffd9b7acc90db989f77e8380f2102c5eb088be02d0d9d08e8"} Aug 04 16:32:11.578447 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.578341 2556 scope.go:117] "RemoveContainer" containerID="7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309" Aug 04 16:32:11.590387 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.590364 2556 scope.go:117] "RemoveContainer" containerID="bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16" Aug 04 16:32:11.593279 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.593234 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" podStartSLOduration=1.593219114 podStartE2EDuration="1.593219114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:32:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:32:11.59168391 +0000 UTC m=+2160.914107394" watchObservedRunningTime="2026-08-04 16:32:11.593219114 +0000 UTC m=+2160.915642595" Aug 04 16:32:11.599777 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.599752 2556 scope.go:117] "RemoveContainer" containerID="7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309" Aug 04 16:32:11.600073 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:32:11.600052 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309\": container with ID starting with 7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309 not found: ID does not exist" containerID="7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309" Aug 04 16:32:11.600174 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.600082 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309"} err="failed to get container status \"7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309\": rpc error: code = NotFound desc = could not find container \"7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309\": container with ID starting with 7f98d1aa989cbdd5458c4312bb43f62557a7c1520df9e38f6c1873d76ae52309 not found: ID does not exist" Aug 04 16:32:11.600174 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.600099 2556 scope.go:117] "RemoveContainer" containerID="bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16" Aug 04 16:32:11.600359 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:32:11.600338 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16\": container with ID starting with bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16 not found: ID does not exist" containerID="bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16" Aug 04 16:32:11.600416 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.600393 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16"} err="failed to get container status \"bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16\": rpc error: code = NotFound desc = could not find container \"bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16\": container with ID starting with bbfdf66b7c40c6a6dc1007fa46cef34a7fe3dc1de1cdc3e08944d4a93ceafb16 not found: ID does not exist" Aug 04 16:32:11.606486 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.606463 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7"] Aug 04 16:32:11.610000 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.609977 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1978158b-openshift-default-78f5d56dd6-fhql7"] Aug 04 16:32:11.628822 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:11.628760 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" podStartSLOduration=1.628744321 podStartE2EDuration="1.628744321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:32:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:32:11.626312158 +0000 UTC m=+2160.948735684" watchObservedRunningTime="2026-08-04 16:32:11.628744321 +0000 UTC m=+2160.951167793" Aug 04 16:32:13.275565 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:13.275523 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e965fb85-d997-4355-94b6-c33029e5021f" path="/var/lib/kubelet/pods/e965fb85-d997-4355-94b6-c33029e5021f/volumes" Aug 04 16:32:22.588979 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:22.588931 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:22.589491 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:22.589059 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:23.557904 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:23.557866 2556 patch_prober.go:28] interesting pod/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 16:32:23.558121 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:23.557940 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 16:32:23.654924 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:23.654884 2556 generic.go:356] "Generic (PLEG): container finished" podID="b383e43e-5c80-40b8-883c-280776d08f20" containerID="6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206" exitCode=0 Aug 04 16:32:23.655415 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:23.654961 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" event={"ID":"b383e43e-5c80-40b8-883c-280776d08f20","Type":"ContainerDied","Data":"6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206"} Aug 04 16:32:23.655415 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:23.655395 2556 scope.go:117] "RemoveContainer" containerID="6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206" Aug 04 16:32:23.656459 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:23.656430 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:32:23.656579 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:23.656563 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:32:23.656643 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:23.656601 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:32:24.663157 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:24.663124 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" event={"ID":"b383e43e-5c80-40b8-883c-280776d08f20","Type":"ContainerStarted","Data":"093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192"} Aug 04 16:32:25.069018 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:25.068991 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:25.069230 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:25.069203 2556 patch_prober.go:28] interesting pod/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 16:32:25.069287 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:25.069268 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 16:32:26.069101 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:26.069058 2556 patch_prober.go:28] interesting pod/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 16:32:26.069590 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:26.069122 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 16:32:27.068875 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:27.068833 2556 patch_prober.go:28] interesting pod/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 16:32:27.069061 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:27.068893 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 16:32:28.068799 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:28.068764 2556 patch_prober.go:28] interesting pod/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 16:32:28.069300 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:28.069219 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 16:32:29.069423 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:29.069387 2556 patch_prober.go:28] interesting pod/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 16:32:29.069942 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:29.069451 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 16:32:30.073348 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:30.073315 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:30.073748 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:30.073692 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:30.074507 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:30.074484 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:40.643697 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:40.643663 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6"] Aug 04 16:32:40.644270 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:40.643895 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" podUID="381eb171-64e1-4f69-8420-a4243f2f91fd" containerName="payload-processing" containerID="cri-o://e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df" gracePeriod=30 Aug 04 16:32:40.651529 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:40.651475 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c"] Aug 04 16:32:40.651811 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:40.651782 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" podUID="26640232-107e-447e-af20-d65af4be88f4" containerName="payload-pre-processing" containerID="cri-o://03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e" gracePeriod=30 Aug 04 16:32:40.909834 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:40.909761 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:40.948563 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:40.948527 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:40.987958 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:40.977687 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk"] Aug 04 16:32:40.987958 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:40.978059 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" containerID="cri-o://093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192" gracePeriod=30 Aug 04 16:32:41.089835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.089805 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4kmz\" (UniqueName: \"kubernetes.io/projected/381eb171-64e1-4f69-8420-a4243f2f91fd-kube-api-access-q4kmz\") pod \"381eb171-64e1-4f69-8420-a4243f2f91fd\" (UID: \"381eb171-64e1-4f69-8420-a4243f2f91fd\") " Aug 04 16:32:41.090000 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.089890 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpkgf\" (UniqueName: \"kubernetes.io/projected/26640232-107e-447e-af20-d65af4be88f4-kube-api-access-zpkgf\") pod \"26640232-107e-447e-af20-d65af4be88f4\" (UID: \"26640232-107e-447e-af20-d65af4be88f4\") " Aug 04 16:32:41.090000 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.089924 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26640232-107e-447e-af20-d65af4be88f4-plugins-config-volume\") pod \"26640232-107e-447e-af20-d65af4be88f4\" (UID: \"26640232-107e-447e-af20-d65af4be88f4\") " Aug 04 16:32:41.090000 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.089945 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/381eb171-64e1-4f69-8420-a4243f2f91fd-plugins-config-volume\") pod \"381eb171-64e1-4f69-8420-a4243f2f91fd\" (UID: \"381eb171-64e1-4f69-8420-a4243f2f91fd\") " Aug 04 16:32:41.090333 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.090295 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/381eb171-64e1-4f69-8420-a4243f2f91fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "381eb171-64e1-4f69-8420-a4243f2f91fd" (UID: "381eb171-64e1-4f69-8420-a4243f2f91fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:32:41.090452 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.090310 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26640232-107e-447e-af20-d65af4be88f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26640232-107e-447e-af20-d65af4be88f4" (UID: "26640232-107e-447e-af20-d65af4be88f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:32:41.092020 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.091999 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/381eb171-64e1-4f69-8420-a4243f2f91fd-kube-api-access-q4kmz" (OuterVolumeSpecName: "kube-api-access-q4kmz") pod "381eb171-64e1-4f69-8420-a4243f2f91fd" (UID: "381eb171-64e1-4f69-8420-a4243f2f91fd"). InnerVolumeSpecName "kube-api-access-q4kmz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:32:41.092102 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.092025 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26640232-107e-447e-af20-d65af4be88f4-kube-api-access-zpkgf" (OuterVolumeSpecName: "kube-api-access-zpkgf") pod "26640232-107e-447e-af20-d65af4be88f4" (UID: "26640232-107e-447e-af20-d65af4be88f4"). InnerVolumeSpecName "kube-api-access-zpkgf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:32:41.190733 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.190632 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpkgf\" (UniqueName: \"kubernetes.io/projected/26640232-107e-447e-af20-d65af4be88f4-kube-api-access-zpkgf\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:41.190733 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.190663 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26640232-107e-447e-af20-d65af4be88f4-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:41.190733 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.190673 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/381eb171-64e1-4f69-8420-a4243f2f91fd-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:41.190733 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.190682 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4kmz\" (UniqueName: \"kubernetes.io/projected/381eb171-64e1-4f69-8420-a4243f2f91fd-kube-api-access-q4kmz\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:41.760037 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.759999 2556 generic.go:356] "Generic (PLEG): container finished" podID="26640232-107e-447e-af20-d65af4be88f4" containerID="03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e" exitCode=0 Aug 04 16:32:41.760527 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.760065 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" Aug 04 16:32:41.760527 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.760092 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" event={"ID":"26640232-107e-447e-af20-d65af4be88f4","Type":"ContainerDied","Data":"03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e"} Aug 04 16:32:41.760527 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.760161 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c" event={"ID":"26640232-107e-447e-af20-d65af4be88f4","Type":"ContainerDied","Data":"b7fa048eb682867928f240b4a083c1c7a42faeed93eeaf6113c28d6ab7f92340"} Aug 04 16:32:41.760527 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.760194 2556 scope.go:117] "RemoveContainer" containerID="03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e" Aug 04 16:32:41.761585 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.761372 2556 generic.go:356] "Generic (PLEG): container finished" podID="381eb171-64e1-4f69-8420-a4243f2f91fd" containerID="e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df" exitCode=0 Aug 04 16:32:41.761585 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.761436 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" Aug 04 16:32:41.761585 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.761441 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" event={"ID":"381eb171-64e1-4f69-8420-a4243f2f91fd","Type":"ContainerDied","Data":"e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df"} Aug 04 16:32:41.761585 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.761483 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6" event={"ID":"381eb171-64e1-4f69-8420-a4243f2f91fd","Type":"ContainerDied","Data":"b6276d64e8b01dbb033969d34829cec1a4dbf9a371cf48dc2979fe735c395d1a"} Aug 04 16:32:41.769499 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.769482 2556 scope.go:117] "RemoveContainer" containerID="03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e" Aug 04 16:32:41.769790 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:32:41.769772 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e\": container with ID starting with 03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e not found: ID does not exist" containerID="03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e" Aug 04 16:32:41.769852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.769796 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e"} err="failed to get container status \"03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e\": rpc error: code = NotFound desc = could not find container \"03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e\": container with ID starting with 03ffa05a82b59451c34a21ad7a237b0eaa0c1a32a0b9e1ee333e3c2180cb730e not found: ID does not exist" Aug 04 16:32:41.769852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.769814 2556 scope.go:117] "RemoveContainer" containerID="e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df" Aug 04 16:32:41.778296 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.778271 2556 scope.go:117] "RemoveContainer" containerID="e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df" Aug 04 16:32:41.778577 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:32:41.778558 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df\": container with ID starting with e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df not found: ID does not exist" containerID="e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df" Aug 04 16:32:41.778625 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.778584 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df"} err="failed to get container status \"e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df\": rpc error: code = NotFound desc = could not find container \"e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df\": container with ID starting with e7fc9d1fdbda5557d781bbc532d9edf446def7cf21806bd6f86d23689318f1df not found: ID does not exist" Aug 04 16:32:41.788347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.788318 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6"] Aug 04 16:32:41.793012 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.792990 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0eb193d3-776b48fc4-mt5b6"] Aug 04 16:32:41.826514 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.826482 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c"] Aug 04 16:32:41.838412 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:41.838383 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0eb193d3-c97f65cf6-9lx8c"] Aug 04 16:32:43.273343 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:43.273298 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26640232-107e-447e-af20-d65af4be88f4" path="/var/lib/kubelet/pods/26640232-107e-447e-af20-d65af4be88f4/volumes" Aug 04 16:32:43.273910 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:43.273889 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="381eb171-64e1-4f69-8420-a4243f2f91fd" path="/var/lib/kubelet/pods/381eb171-64e1-4f69-8420-a4243f2f91fd/volumes" Aug 04 16:32:45.076249 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:45.076209 2556 patch_prober.go:28] interesting pod/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 16:32:45.076626 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:45.076269 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:32:46.246369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.246341 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:46.335695 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.335608 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b383e43e-5c80-40b8-883c-280776d08f20-istio-podinfo\") pod \"b383e43e-5c80-40b8-883c-280776d08f20\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " Aug 04 16:32:46.335914 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.335705 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-istio-token\") pod \"b383e43e-5c80-40b8-883c-280776d08f20\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " Aug 04 16:32:46.335914 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.335812 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-data\") pod \"b383e43e-5c80-40b8-883c-280776d08f20\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " Aug 04 16:32:46.335914 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.335871 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-credential-socket\") pod \"b383e43e-5c80-40b8-883c-280776d08f20\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " Aug 04 16:32:46.335914 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.335895 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-certs\") pod \"b383e43e-5c80-40b8-883c-280776d08f20\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " Aug 04 16:32:46.336129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.335919 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-socket\") pod \"b383e43e-5c80-40b8-883c-280776d08f20\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " Aug 04 16:32:46.336129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.335959 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b383e43e-5c80-40b8-883c-280776d08f20-istiod-ca-cert\") pod \"b383e43e-5c80-40b8-883c-280776d08f20\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " Aug 04 16:32:46.336129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.336030 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rbdz\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-kube-api-access-6rbdz\") pod \"b383e43e-5c80-40b8-883c-280776d08f20\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " Aug 04 16:32:46.336129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.336075 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-envoy\") pod \"b383e43e-5c80-40b8-883c-280776d08f20\" (UID: \"b383e43e-5c80-40b8-883c-280776d08f20\") " Aug 04 16:32:46.336129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.336111 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-data" (OuterVolumeSpecName: "istio-data") pod "b383e43e-5c80-40b8-883c-280776d08f20" (UID: "b383e43e-5c80-40b8-883c-280776d08f20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:46.336331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.336130 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b383e43e-5c80-40b8-883c-280776d08f20" (UID: "b383e43e-5c80-40b8-883c-280776d08f20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:46.336505 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.336477 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b383e43e-5c80-40b8-883c-280776d08f20" (UID: "b383e43e-5c80-40b8-883c-280776d08f20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:46.338504 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.336935 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:46.338504 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.336965 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:46.338504 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.336981 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:46.338504 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.337134 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b383e43e-5c80-40b8-883c-280776d08f20" (UID: "b383e43e-5c80-40b8-883c-280776d08f20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:46.338504 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.337171 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b383e43e-5c80-40b8-883c-280776d08f20-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b383e43e-5c80-40b8-883c-280776d08f20" (UID: "b383e43e-5c80-40b8-883c-280776d08f20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:32:46.338504 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.338389 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b383e43e-5c80-40b8-883c-280776d08f20-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b383e43e-5c80-40b8-883c-280776d08f20" (UID: "b383e43e-5c80-40b8-883c-280776d08f20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:32:46.343783 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.339249 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b383e43e-5c80-40b8-883c-280776d08f20" (UID: "b383e43e-5c80-40b8-883c-280776d08f20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:32:46.343783 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.339305 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-istio-token" (OuterVolumeSpecName: "istio-token") pod "b383e43e-5c80-40b8-883c-280776d08f20" (UID: "b383e43e-5c80-40b8-883c-280776d08f20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:32:46.345168 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.345138 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-kube-api-access-6rbdz" (OuterVolumeSpecName: "kube-api-access-6rbdz") pod "b383e43e-5c80-40b8-883c-280776d08f20" (UID: "b383e43e-5c80-40b8-883c-280776d08f20"). InnerVolumeSpecName "kube-api-access-6rbdz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:32:46.437852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.437816 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:46.437852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.437847 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b383e43e-5c80-40b8-883c-280776d08f20-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:46.437852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.437857 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rbdz\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-kube-api-access-6rbdz\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:46.438087 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.437866 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b383e43e-5c80-40b8-883c-280776d08f20-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:46.438087 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.437874 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b383e43e-5c80-40b8-883c-280776d08f20-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:46.438087 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.437882 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b383e43e-5c80-40b8-883c-280776d08f20-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:32:46.791085 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.791045 2556 generic.go:356] "Generic (PLEG): container finished" podID="b383e43e-5c80-40b8-883c-280776d08f20" containerID="093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192" exitCode=0 Aug 04 16:32:46.791289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.791118 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" Aug 04 16:32:46.791289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.791133 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" event={"ID":"b383e43e-5c80-40b8-883c-280776d08f20","Type":"ContainerDied","Data":"093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192"} Aug 04 16:32:46.791289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.791248 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk" event={"ID":"b383e43e-5c80-40b8-883c-280776d08f20","Type":"ContainerDied","Data":"72ea5aedaafd51da8eb1a1071eed9b2d409ec2b76563d4d923dea03114283384"} Aug 04 16:32:46.791289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.791280 2556 scope.go:117] "RemoveContainer" containerID="093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192" Aug 04 16:32:46.800536 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.800514 2556 scope.go:117] "RemoveContainer" containerID="6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206" Aug 04 16:32:46.809002 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.808982 2556 scope.go:117] "RemoveContainer" containerID="093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192" Aug 04 16:32:46.809266 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:32:46.809245 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192\": container with ID starting with 093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192 not found: ID does not exist" containerID="093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192" Aug 04 16:32:46.809311 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.809276 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192"} err="failed to get container status \"093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192\": rpc error: code = NotFound desc = could not find container \"093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192\": container with ID starting with 093eecce51027e8e9de35b712372b53fbc7c4c84287b2d4049ab010d81cf4192 not found: ID does not exist" Aug 04 16:32:46.809311 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.809296 2556 scope.go:117] "RemoveContainer" containerID="6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206" Aug 04 16:32:46.809559 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:32:46.809537 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206\": container with ID starting with 6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206 not found: ID does not exist" containerID="6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206" Aug 04 16:32:46.809613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.809566 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206"} err="failed to get container status \"6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206\": rpc error: code = NotFound desc = could not find container \"6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206\": container with ID starting with 6806b31a5c6c887e82e78ba97948709876ebb5d38ebdc40fa30db39c47f3a206 not found: ID does not exist" Aug 04 16:32:46.814523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.814496 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk"] Aug 04 16:32:46.818115 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:46.818091 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-0eb193d3-openshift-default-59cbcc4f6-qmhmk"] Aug 04 16:32:47.273338 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:32:47.273295 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b383e43e-5c80-40b8-883c-280776d08f20" path="/var/lib/kubelet/pods/b383e43e-5c80-40b8-883c-280776d08f20/volumes" Aug 04 16:34:40.608782 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.608746 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk"] Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609199 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609213 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609228 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="381eb171-64e1-4f69-8420-a4243f2f91fd" containerName="payload-processing" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609234 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="381eb171-64e1-4f69-8420-a4243f2f91fd" containerName="payload-processing" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609245 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609255 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609265 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609270 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609279 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26640232-107e-447e-af20-d65af4be88f4" containerName="payload-pre-processing" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609285 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="26640232-107e-447e-af20-d65af4be88f4" containerName="payload-pre-processing" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609298 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" Aug 04 16:34:40.609352 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609303 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" Aug 04 16:34:40.609978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609367 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" Aug 04 16:34:40.609978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609374 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" Aug 04 16:34:40.609978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609382 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b383e43e-5c80-40b8-883c-280776d08f20" containerName="istio-proxy" Aug 04 16:34:40.609978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609388 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="26640232-107e-447e-af20-d65af4be88f4" containerName="payload-pre-processing" Aug 04 16:34:40.609978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609400 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="381eb171-64e1-4f69-8420-a4243f2f91fd" containerName="payload-processing" Aug 04 16:34:40.609978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.609410 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e965fb85-d997-4355-94b6-c33029e5021f" containerName="istio-proxy" Aug 04 16:34:40.612781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.612765 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.615768 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.615743 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7c9a38ca-openshift-default-dockercfg-nccpx\"" Aug 04 16:34:40.627741 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.627701 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk"] Aug 04 16:34:40.726850 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.726813 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-token\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.727031 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.726855 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-credential-socket\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.727031 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.726882 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-podinfo\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.727031 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.726940 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-certs\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.727031 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.726981 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istiod-ca-cert\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.727175 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.727070 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-data\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.727175 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.727110 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25rpv\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-kube-api-access-25rpv\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.727175 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.727132 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-envoy\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.727175 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.727165 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-socket\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.828385 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828345 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istiod-ca-cert\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.828578 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828408 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-data\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.828578 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828435 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25rpv\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-kube-api-access-25rpv\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.828578 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828455 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-envoy\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.828578 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828483 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-socket\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.828578 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828512 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-token\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.828912 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828676 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-credential-socket\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.828912 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828741 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-podinfo\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.828912 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828781 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-certs\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.829058 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828913 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-data\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.829058 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.828997 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-socket\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.829153 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.829127 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-certs\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.829229 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.829202 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-credential-socket\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.829270 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.829217 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istiod-ca-cert\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.830828 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.830809 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-envoy\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.831132 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.831112 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-podinfo\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.837054 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.837032 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-token\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.839898 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.839875 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25rpv\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-kube-api-access-25rpv\") pod \"e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:40.924398 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:40.924318 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:41.073946 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:41.073917 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk"] Aug 04 16:34:41.075458 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:34:41.075430 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e221616_cf85_48ca_a31f_e1a7f5d75b37.slice/crio-8bcf555e5d03aaf3c6401927f7a4dfedbe1614528f36ee6a7b1a7b26535f6756 WatchSource:0}: Error finding container 8bcf555e5d03aaf3c6401927f7a4dfedbe1614528f36ee6a7b1a7b26535f6756: Status 404 returned error can't find the container with id 8bcf555e5d03aaf3c6401927f7a4dfedbe1614528f36ee6a7b1a7b26535f6756 Aug 04 16:34:41.077849 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:41.077818 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:34:41.077929 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:41.077881 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:34:41.077929 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:41.077910 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:34:41.374633 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:41.374597 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" event={"ID":"9e221616-cf85-48ca-a31f-e1a7f5d75b37","Type":"ContainerStarted","Data":"caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99"} Aug 04 16:34:41.374869 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:41.374657 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" event={"ID":"9e221616-cf85-48ca-a31f-e1a7f5d75b37","Type":"ContainerStarted","Data":"8bcf555e5d03aaf3c6401927f7a4dfedbe1614528f36ee6a7b1a7b26535f6756"} Aug 04 16:34:41.396588 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:41.396529 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" podStartSLOduration=1.396512775 podStartE2EDuration="1.396512775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:34:41.394978914 +0000 UTC m=+2310.717402394" watchObservedRunningTime="2026-08-04 16:34:41.396512775 +0000 UTC m=+2310.718936255" Aug 04 16:34:41.925282 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:41.925254 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:42.929899 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:42.929868 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:43.386136 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:43.386107 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:43.387035 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:43.387011 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:45.580204 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.580176 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4"] Aug 04 16:34:45.583908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.583892 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:45.587369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.587345 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7c9a38ca-dockercfg-l42fw\"" Aug 04 16:34:45.587782 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.587766 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7c9a38ca\"" Aug 04 16:34:45.596556 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.596535 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4"] Aug 04 16:34:45.665558 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.665522 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj"] Aug 04 16:34:45.669072 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.669055 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:45.680678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.680655 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfdg4\" (UniqueName: \"kubernetes.io/projected/a12bf065-4b59-4277-a461-7e11876d8911-kube-api-access-zfdg4\") pod \"payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj\" (UID: \"a12bf065-4b59-4277-a461-7e11876d8911\") " pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:45.680845 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.680694 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12bf065-4b59-4277-a461-7e11876d8911-plugins-config-volume\") pod \"payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj\" (UID: \"a12bf065-4b59-4277-a461-7e11876d8911\") " pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:45.680845 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.680716 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e5415d-d8e9-4f05-b657-a992957e604c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4\" (UID: \"93e5415d-d8e9-4f05-b657-a992957e604c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:45.680924 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.680837 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdmc2\" (UniqueName: \"kubernetes.io/projected/93e5415d-d8e9-4f05-b657-a992957e604c-kube-api-access-bdmc2\") pod \"payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4\" (UID: \"93e5415d-d8e9-4f05-b657-a992957e604c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:45.693401 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.693369 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj"] Aug 04 16:34:45.781904 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.781870 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfdg4\" (UniqueName: \"kubernetes.io/projected/a12bf065-4b59-4277-a461-7e11876d8911-kube-api-access-zfdg4\") pod \"payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj\" (UID: \"a12bf065-4b59-4277-a461-7e11876d8911\") " pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:45.782096 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.781926 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12bf065-4b59-4277-a461-7e11876d8911-plugins-config-volume\") pod \"payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj\" (UID: \"a12bf065-4b59-4277-a461-7e11876d8911\") " pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:45.782096 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.781957 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e5415d-d8e9-4f05-b657-a992957e604c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4\" (UID: \"93e5415d-d8e9-4f05-b657-a992957e604c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:45.782096 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.782066 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdmc2\" (UniqueName: \"kubernetes.io/projected/93e5415d-d8e9-4f05-b657-a992957e604c-kube-api-access-bdmc2\") pod \"payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4\" (UID: \"93e5415d-d8e9-4f05-b657-a992957e604c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:45.782708 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.782683 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12bf065-4b59-4277-a461-7e11876d8911-plugins-config-volume\") pod \"payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj\" (UID: \"a12bf065-4b59-4277-a461-7e11876d8911\") " pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:45.782708 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.782701 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e5415d-d8e9-4f05-b657-a992957e604c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4\" (UID: \"93e5415d-d8e9-4f05-b657-a992957e604c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:45.794568 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.794530 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdmc2\" (UniqueName: \"kubernetes.io/projected/93e5415d-d8e9-4f05-b657-a992957e604c-kube-api-access-bdmc2\") pod \"payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4\" (UID: \"93e5415d-d8e9-4f05-b657-a992957e604c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:45.794568 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.794546 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfdg4\" (UniqueName: \"kubernetes.io/projected/a12bf065-4b59-4277-a461-7e11876d8911-kube-api-access-zfdg4\") pod \"payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj\" (UID: \"a12bf065-4b59-4277-a461-7e11876d8911\") " pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:45.894951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.894862 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:45.979610 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:45.979560 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:46.052663 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.052634 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4"] Aug 04 16:34:46.054596 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:34:46.054538 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e5415d_d8e9_4f05_b657_a992957e604c.slice/crio-2956717ebb93501401ca98a3eca2856918839805ad7e22be305f321331652a0a WatchSource:0}: Error finding container 2956717ebb93501401ca98a3eca2856918839805ad7e22be305f321331652a0a: Status 404 returned error can't find the container with id 2956717ebb93501401ca98a3eca2856918839805ad7e22be305f321331652a0a Aug 04 16:34:46.141491 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.141466 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj"] Aug 04 16:34:46.143045 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:34:46.143018 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12bf065_4b59_4277_a461_7e11876d8911.slice/crio-9d5326d1018c268f8a9952e28e6c0d2ad6de42bc0a44ad63937a6c8c7e56ac51 WatchSource:0}: Error finding container 9d5326d1018c268f8a9952e28e6c0d2ad6de42bc0a44ad63937a6c8c7e56ac51: Status 404 returned error can't find the container with id 9d5326d1018c268f8a9952e28e6c0d2ad6de42bc0a44ad63937a6c8c7e56ac51 Aug 04 16:34:46.403096 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.402996 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" event={"ID":"93e5415d-d8e9-4f05-b657-a992957e604c","Type":"ContainerStarted","Data":"c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161"} Aug 04 16:34:46.403096 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.403066 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:46.403096 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.403093 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" event={"ID":"93e5415d-d8e9-4f05-b657-a992957e604c","Type":"ContainerStarted","Data":"2956717ebb93501401ca98a3eca2856918839805ad7e22be305f321331652a0a"} Aug 04 16:34:46.404531 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.404502 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" event={"ID":"a12bf065-4b59-4277-a461-7e11876d8911","Type":"ContainerStarted","Data":"be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407"} Aug 04 16:34:46.404650 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.404541 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" event={"ID":"a12bf065-4b59-4277-a461-7e11876d8911","Type":"ContainerStarted","Data":"9d5326d1018c268f8a9952e28e6c0d2ad6de42bc0a44ad63937a6c8c7e56ac51"} Aug 04 16:34:46.404650 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.404570 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:46.425539 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.425484 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" podStartSLOduration=1.425468326 podStartE2EDuration="1.425468326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:34:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:34:46.42421894 +0000 UTC m=+2315.746642420" watchObservedRunningTime="2026-08-04 16:34:46.425468326 +0000 UTC m=+2315.747891806" Aug 04 16:34:46.452593 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:46.452538 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" podStartSLOduration=1.452523128 podStartE2EDuration="1.452523128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:34:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:34:46.449578037 +0000 UTC m=+2315.772001518" watchObservedRunningTime="2026-08-04 16:34:46.452523128 +0000 UTC m=+2315.774946608" Aug 04 16:34:57.411523 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:57.411495 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:34:57.411955 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:57.411568 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:34:57.463830 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:57.463752 2556 generic.go:356] "Generic (PLEG): container finished" podID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerID="caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99" exitCode=0 Aug 04 16:34:57.463978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:57.463816 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" event={"ID":"9e221616-cf85-48ca-a31f-e1a7f5d75b37","Type":"ContainerDied","Data":"caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99"} Aug 04 16:34:57.464382 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:57.464361 2556 scope.go:117] "RemoveContainer" containerID="caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99" Aug 04 16:34:57.465690 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:57.465663 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:34:57.465842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:57.465774 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:34:57.465842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:57.465827 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:34:58.386953 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:58.386901 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:58.470696 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:58.470659 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" event={"ID":"9e221616-cf85-48ca-a31f-e1a7f5d75b37","Type":"ContainerStarted","Data":"abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59"} Aug 04 16:34:58.925116 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:58.925080 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:34:58.926010 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:58.925982 2556 patch_prober.go:28] interesting pod/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Aug 04 16:34:58.926125 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:58.926032 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Aug 04 16:34:59.925528 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:59.925491 2556 patch_prober.go:28] interesting pod/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Aug 04 16:34:59.925957 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:34:59.925544 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Aug 04 16:35:00.925287 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:00.925246 2556 patch_prober.go:28] interesting pod/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Aug 04 16:35:00.925457 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:00.925320 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Aug 04 16:35:01.925273 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:01.925240 2556 patch_prober.go:28] interesting pod/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Aug 04 16:35:01.925654 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:01.925292 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Aug 04 16:35:02.925556 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:02.925518 2556 patch_prober.go:28] interesting pod/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Aug 04 16:35:02.925973 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:02.925572 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Aug 04 16:35:03.930048 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:03.930014 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:35:03.930450 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:03.930430 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:35:03.931108 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:03.931087 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:35:26.817312 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:26.817278 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj"] Aug 04 16:35:26.817861 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:26.817536 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" podUID="a12bf065-4b59-4277-a461-7e11876d8911" containerName="payload-processing" containerID="cri-o://be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407" gracePeriod=30 Aug 04 16:35:26.826734 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:26.826683 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4"] Aug 04 16:35:26.826989 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:26.826940 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" podUID="93e5415d-d8e9-4f05-b657-a992957e604c" containerName="payload-pre-processing" containerID="cri-o://c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161" gracePeriod=30 Aug 04 16:35:27.093961 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.093931 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:35:27.124902 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.124877 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:35:27.192390 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.192354 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfdg4\" (UniqueName: \"kubernetes.io/projected/a12bf065-4b59-4277-a461-7e11876d8911-kube-api-access-zfdg4\") pod \"a12bf065-4b59-4277-a461-7e11876d8911\" (UID: \"a12bf065-4b59-4277-a461-7e11876d8911\") " Aug 04 16:35:27.192598 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.192411 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12bf065-4b59-4277-a461-7e11876d8911-plugins-config-volume\") pod \"a12bf065-4b59-4277-a461-7e11876d8911\" (UID: \"a12bf065-4b59-4277-a461-7e11876d8911\") " Aug 04 16:35:27.192598 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.192456 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e5415d-d8e9-4f05-b657-a992957e604c-plugins-config-volume\") pod \"93e5415d-d8e9-4f05-b657-a992957e604c\" (UID: \"93e5415d-d8e9-4f05-b657-a992957e604c\") " Aug 04 16:35:27.192598 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.192538 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdmc2\" (UniqueName: \"kubernetes.io/projected/93e5415d-d8e9-4f05-b657-a992957e604c-kube-api-access-bdmc2\") pod \"93e5415d-d8e9-4f05-b657-a992957e604c\" (UID: \"93e5415d-d8e9-4f05-b657-a992957e604c\") " Aug 04 16:35:27.192890 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.192857 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e5415d-d8e9-4f05-b657-a992957e604c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93e5415d-d8e9-4f05-b657-a992957e604c" (UID: "93e5415d-d8e9-4f05-b657-a992957e604c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:35:27.192890 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.192876 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a12bf065-4b59-4277-a461-7e11876d8911-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a12bf065-4b59-4277-a461-7e11876d8911" (UID: "a12bf065-4b59-4277-a461-7e11876d8911"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:35:27.194613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.194592 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e5415d-d8e9-4f05-b657-a992957e604c-kube-api-access-bdmc2" (OuterVolumeSpecName: "kube-api-access-bdmc2") pod "93e5415d-d8e9-4f05-b657-a992957e604c" (UID: "93e5415d-d8e9-4f05-b657-a992957e604c"). InnerVolumeSpecName "kube-api-access-bdmc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:35:27.194680 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.194613 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a12bf065-4b59-4277-a461-7e11876d8911-kube-api-access-zfdg4" (OuterVolumeSpecName: "kube-api-access-zfdg4") pod "a12bf065-4b59-4277-a461-7e11876d8911" (UID: "a12bf065-4b59-4277-a461-7e11876d8911"). InnerVolumeSpecName "kube-api-access-zfdg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:35:27.294084 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.294055 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e5415d-d8e9-4f05-b657-a992957e604c-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:27.294084 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.294080 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdmc2\" (UniqueName: \"kubernetes.io/projected/93e5415d-d8e9-4f05-b657-a992957e604c-kube-api-access-bdmc2\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:27.294247 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.294089 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfdg4\" (UniqueName: \"kubernetes.io/projected/a12bf065-4b59-4277-a461-7e11876d8911-kube-api-access-zfdg4\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:27.294247 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.294099 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12bf065-4b59-4277-a461-7e11876d8911-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:27.642303 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.642213 2556 generic.go:356] "Generic (PLEG): container finished" podID="93e5415d-d8e9-4f05-b657-a992957e604c" containerID="c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161" exitCode=0 Aug 04 16:35:27.642491 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.642261 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" Aug 04 16:35:27.642491 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.642277 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" event={"ID":"93e5415d-d8e9-4f05-b657-a992957e604c","Type":"ContainerDied","Data":"c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161"} Aug 04 16:35:27.642615 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.642529 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4" event={"ID":"93e5415d-d8e9-4f05-b657-a992957e604c","Type":"ContainerDied","Data":"2956717ebb93501401ca98a3eca2856918839805ad7e22be305f321331652a0a"} Aug 04 16:35:27.642615 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.642570 2556 scope.go:117] "RemoveContainer" containerID="c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161" Aug 04 16:35:27.643762 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.643740 2556 generic.go:356] "Generic (PLEG): container finished" podID="a12bf065-4b59-4277-a461-7e11876d8911" containerID="be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407" exitCode=0 Aug 04 16:35:27.643840 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.643791 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" Aug 04 16:35:27.643840 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.643816 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" event={"ID":"a12bf065-4b59-4277-a461-7e11876d8911","Type":"ContainerDied","Data":"be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407"} Aug 04 16:35:27.643925 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.643869 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj" event={"ID":"a12bf065-4b59-4277-a461-7e11876d8911","Type":"ContainerDied","Data":"9d5326d1018c268f8a9952e28e6c0d2ad6de42bc0a44ad63937a6c8c7e56ac51"} Aug 04 16:35:27.651842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.651825 2556 scope.go:117] "RemoveContainer" containerID="c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161" Aug 04 16:35:27.652109 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:35:27.652087 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161\": container with ID starting with c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161 not found: ID does not exist" containerID="c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161" Aug 04 16:35:27.652165 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.652118 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161"} err="failed to get container status \"c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161\": rpc error: code = NotFound desc = could not find container \"c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161\": container with ID starting with c9ebfc59f417de3854d99ce9f98ea22b69fc1e293b80448a6c21a564fc121161 not found: ID does not exist" Aug 04 16:35:27.652165 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.652137 2556 scope.go:117] "RemoveContainer" containerID="be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407" Aug 04 16:35:27.660477 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.660458 2556 scope.go:117] "RemoveContainer" containerID="be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407" Aug 04 16:35:27.660759 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:35:27.660714 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407\": container with ID starting with be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407 not found: ID does not exist" containerID="be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407" Aug 04 16:35:27.660853 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.660767 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407"} err="failed to get container status \"be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407\": rpc error: code = NotFound desc = could not find container \"be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407\": container with ID starting with be0cbc0b0f6d0802f689d336ee53af4eb85ebfe730f9755175ada163dae9c407 not found: ID does not exist" Aug 04 16:35:27.670925 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.670901 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4"] Aug 04 16:35:27.676894 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.676860 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7c9a38ca-7948f5d544-97wt4"] Aug 04 16:35:27.692836 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.692801 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj"] Aug 04 16:35:27.698161 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:27.698136 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c9a38ca-675cd85cd-2ldxj"] Aug 04 16:35:29.274027 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:29.273988 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93e5415d-d8e9-4f05-b657-a992957e604c" path="/var/lib/kubelet/pods/93e5415d-d8e9-4f05-b657-a992957e604c/volumes" Aug 04 16:35:29.274571 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:29.274550 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a12bf065-4b59-4277-a461-7e11876d8911" path="/var/lib/kubelet/pods/a12bf065-4b59-4277-a461-7e11876d8911/volumes" Aug 04 16:35:33.977902 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:33.977865 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk"] Aug 04 16:35:33.978378 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:33.978161 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" containerID="cri-o://abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59" gracePeriod=30 Aug 04 16:35:39.233572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.233542 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:35:39.306678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.306596 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-podinfo\") pod \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " Aug 04 16:35:39.306678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.306635 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25rpv\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-kube-api-access-25rpv\") pod \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " Aug 04 16:35:39.306678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.306678 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-data\") pod \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " Aug 04 16:35:39.306993 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.306713 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-certs\") pod \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " Aug 04 16:35:39.306993 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.306747 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-socket\") pod \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " Aug 04 16:35:39.306993 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.306793 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istiod-ca-cert\") pod \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " Aug 04 16:35:39.306993 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.306833 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-token\") pod \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " Aug 04 16:35:39.306993 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.306882 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-envoy\") pod \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " Aug 04 16:35:39.306993 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.306915 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-credential-socket\") pod \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\" (UID: \"9e221616-cf85-48ca-a31f-e1a7f5d75b37\") " Aug 04 16:35:39.307318 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.307110 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e221616-cf85-48ca-a31f-e1a7f5d75b37" (UID: "9e221616-cf85-48ca-a31f-e1a7f5d75b37"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:35:39.307318 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.307126 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e221616-cf85-48ca-a31f-e1a7f5d75b37" (UID: "9e221616-cf85-48ca-a31f-e1a7f5d75b37"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:35:39.307318 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.307135 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e221616-cf85-48ca-a31f-e1a7f5d75b37" (UID: "9e221616-cf85-48ca-a31f-e1a7f5d75b37"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:35:39.307318 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.307256 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:39.307318 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.307275 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:39.307577 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.307320 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e221616-cf85-48ca-a31f-e1a7f5d75b37" (UID: "9e221616-cf85-48ca-a31f-e1a7f5d75b37"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:35:39.307577 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.307327 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e221616-cf85-48ca-a31f-e1a7f5d75b37" (UID: "9e221616-cf85-48ca-a31f-e1a7f5d75b37"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:35:39.308964 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.308940 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e221616-cf85-48ca-a31f-e1a7f5d75b37" (UID: "9e221616-cf85-48ca-a31f-e1a7f5d75b37"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:35:39.308964 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.308943 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-kube-api-access-25rpv" (OuterVolumeSpecName: "kube-api-access-25rpv") pod "9e221616-cf85-48ca-a31f-e1a7f5d75b37" (UID: "9e221616-cf85-48ca-a31f-e1a7f5d75b37"). InnerVolumeSpecName "kube-api-access-25rpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:35:39.309144 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.309050 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e221616-cf85-48ca-a31f-e1a7f5d75b37" (UID: "9e221616-cf85-48ca-a31f-e1a7f5d75b37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:35:39.309187 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.309169 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e221616-cf85-48ca-a31f-e1a7f5d75b37" (UID: "9e221616-cf85-48ca-a31f-e1a7f5d75b37"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:35:39.408494 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.408457 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:39.408494 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.408489 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:39.408494 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.408498 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:39.408770 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.408508 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:39.408770 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.408517 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25rpv\" (UniqueName: \"kubernetes.io/projected/9e221616-cf85-48ca-a31f-e1a7f5d75b37-kube-api-access-25rpv\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:39.408770 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.408526 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e221616-cf85-48ca-a31f-e1a7f5d75b37-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:39.408770 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.408535 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e221616-cf85-48ca-a31f-e1a7f5d75b37-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:35:39.706301 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.706211 2556 generic.go:356] "Generic (PLEG): container finished" podID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerID="abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59" exitCode=0 Aug 04 16:35:39.706301 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.706262 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" Aug 04 16:35:39.706471 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.706296 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" event={"ID":"9e221616-cf85-48ca-a31f-e1a7f5d75b37","Type":"ContainerDied","Data":"abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59"} Aug 04 16:35:39.706471 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.706354 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk" event={"ID":"9e221616-cf85-48ca-a31f-e1a7f5d75b37","Type":"ContainerDied","Data":"8bcf555e5d03aaf3c6401927f7a4dfedbe1614528f36ee6a7b1a7b26535f6756"} Aug 04 16:35:39.706471 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.706384 2556 scope.go:117] "RemoveContainer" containerID="abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59" Aug 04 16:35:39.715976 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.715953 2556 scope.go:117] "RemoveContainer" containerID="caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99" Aug 04 16:35:39.724209 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.724182 2556 scope.go:117] "RemoveContainer" containerID="abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59" Aug 04 16:35:39.724496 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:35:39.724475 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59\": container with ID starting with abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59 not found: ID does not exist" containerID="abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59" Aug 04 16:35:39.724549 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.724507 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59"} err="failed to get container status \"abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59\": rpc error: code = NotFound desc = could not find container \"abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59\": container with ID starting with abe719a0677f696ca4282f59007c689448730a0dd87a980e0694ecc8dca96d59 not found: ID does not exist" Aug 04 16:35:39.724549 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.724525 2556 scope.go:117] "RemoveContainer" containerID="caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99" Aug 04 16:35:39.724831 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:35:39.724799 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99\": container with ID starting with caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99 not found: ID does not exist" containerID="caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99" Aug 04 16:35:39.724934 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.724839 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99"} err="failed to get container status \"caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99\": rpc error: code = NotFound desc = could not find container \"caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99\": container with ID starting with caf90d3a6fc69a082076ad97c43b5f01b2b2969ca5e11298fdbf1c10d8e7dc99 not found: ID does not exist" Aug 04 16:35:39.730796 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.730768 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk"] Aug 04 16:35:39.732568 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:39.732544 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7c9a38ca-openshift-default-667bf9b8-ck5tk"] Aug 04 16:35:41.274137 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:35:41.274076 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" path="/var/lib/kubelet/pods/9e221616-cf85-48ca-a31f-e1a7f5d75b37/volumes" Aug 04 16:37:51.357334 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.357298 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz"] Aug 04 16:37:51.358004 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.357986 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" Aug 04 16:37:51.358066 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.358007 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" Aug 04 16:37:51.358066 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.358020 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93e5415d-d8e9-4f05-b657-a992957e604c" containerName="payload-pre-processing" Aug 04 16:37:51.358066 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.358029 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="93e5415d-d8e9-4f05-b657-a992957e604c" containerName="payload-pre-processing" Aug 04 16:37:51.358164 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.358073 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a12bf065-4b59-4277-a461-7e11876d8911" containerName="payload-processing" Aug 04 16:37:51.358164 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.358082 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a12bf065-4b59-4277-a461-7e11876d8911" containerName="payload-processing" Aug 04 16:37:51.358235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.358169 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" Aug 04 16:37:51.358235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.358182 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a12bf065-4b59-4277-a461-7e11876d8911" containerName="payload-processing" Aug 04 16:37:51.358235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.358195 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" Aug 04 16:37:51.358235 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.358205 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="93e5415d-d8e9-4f05-b657-a992957e604c" containerName="payload-pre-processing" Aug 04 16:37:51.361672 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.361651 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.364847 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.364825 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c2a82a29-openshift-default-dockercfg-dspfq\"" Aug 04 16:37:51.374099 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.374073 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz"] Aug 04 16:37:51.430644 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.430608 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-credential-socket\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.430884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.430667 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwpxk\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-kube-api-access-zwpxk\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.430884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.430698 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-socket\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.430884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.430765 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-data\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.430884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.430787 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c83c411-c019-439c-a89f-79ff6fe41186-istio-podinfo\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.430884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.430838 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-envoy\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.430884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.430877 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c83c411-c019-439c-a89f-79ff6fe41186-istiod-ca-cert\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.431155 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.430925 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-istio-token\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.431155 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.430978 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-certs\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.531854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.531819 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c83c411-c019-439c-a89f-79ff6fe41186-istiod-ca-cert\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.531854 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.531856 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-istio-token\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.531884 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-certs\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.531923 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-credential-socket\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.531955 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwpxk\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-kube-api-access-zwpxk\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.531982 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-socket\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.532023 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-data\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.532049 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c83c411-c019-439c-a89f-79ff6fe41186-istio-podinfo\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532394 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.532106 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-envoy\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532394 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.532339 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-certs\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532511 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.532472 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-socket\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532566 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.532519 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-credential-socket\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532617 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.532584 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c83c411-c019-439c-a89f-79ff6fe41186-istiod-ca-cert\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.532617 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.532598 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-data\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.534440 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.534421 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-envoy\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.534567 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.534550 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c83c411-c019-439c-a89f-79ff6fe41186-istio-podinfo\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.545050 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.545025 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-istio-token\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.545144 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.545100 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwpxk\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-kube-api-access-zwpxk\") pod \"e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.676122 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.676028 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:51.825511 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.825479 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz"] Aug 04 16:37:51.827344 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:37:51.827321 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c83c411_c019_439c_a89f_79ff6fe41186.slice/crio-802d1235dc1bacd501fe7c2cccf547581bf9c61bd4c33103b6dd88d1223cb947 WatchSource:0}: Error finding container 802d1235dc1bacd501fe7c2cccf547581bf9c61bd4c33103b6dd88d1223cb947: Status 404 returned error can't find the container with id 802d1235dc1bacd501fe7c2cccf547581bf9c61bd4c33103b6dd88d1223cb947 Aug 04 16:37:51.829454 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.829424 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:37:51.829525 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.829492 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:37:51.829525 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:51.829520 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:37:52.386807 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:52.386768 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" event={"ID":"7c83c411-c019-439c-a89f-79ff6fe41186","Type":"ContainerStarted","Data":"37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44"} Aug 04 16:37:52.387205 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:52.386820 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" event={"ID":"7c83c411-c019-439c-a89f-79ff6fe41186","Type":"ContainerStarted","Data":"802d1235dc1bacd501fe7c2cccf547581bf9c61bd4c33103b6dd88d1223cb947"} Aug 04 16:37:52.416496 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:52.416436 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" podStartSLOduration=1.416421498 podStartE2EDuration="1.416421498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:37:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:37:52.414317909 +0000 UTC m=+2501.736741390" watchObservedRunningTime="2026-08-04 16:37:52.416421498 +0000 UTC m=+2501.738844978" Aug 04 16:37:52.676883 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:52.676780 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:52.681800 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:52.681772 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:53.392260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:53.392223 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:53.393226 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:53.393201 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:37:56.423404 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.423361 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6"] Aug 04 16:37:56.423995 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.423974 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" Aug 04 16:37:56.424076 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.423997 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e221616-cf85-48ca-a31f-e1a7f5d75b37" containerName="istio-proxy" Aug 04 16:37:56.427255 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.427232 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:37:56.430320 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.430291 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c2a82a29\"" Aug 04 16:37:56.430459 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.430433 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c2a82a29-dockercfg-hrfqd\"" Aug 04 16:37:56.436387 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.436359 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6"] Aug 04 16:37:56.484358 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.484319 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xcns\" (UniqueName: \"kubernetes.io/projected/9de13670-914f-4618-a41f-f9c2124f33c2-kube-api-access-8xcns\") pod \"payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6\" (UID: \"9de13670-914f-4618-a41f-f9c2124f33c2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:37:56.484520 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.484367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9de13670-914f-4618-a41f-f9c2124f33c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6\" (UID: \"9de13670-914f-4618-a41f-f9c2124f33c2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:37:56.513359 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.513172 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68"] Aug 04 16:37:56.518142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.518117 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:37:56.529652 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.529576 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68"] Aug 04 16:37:56.585436 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.585405 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n94zz\" (UniqueName: \"kubernetes.io/projected/1b36ca06-3d37-4ecb-8ac8-efde060d3115-kube-api-access-n94zz\") pod \"payload-processing-e2e-mt-c2a82a29-577d456474-hbc68\" (UID: \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\") " pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:37:56.585638 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.585520 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b36ca06-3d37-4ecb-8ac8-efde060d3115-plugins-config-volume\") pod \"payload-processing-e2e-mt-c2a82a29-577d456474-hbc68\" (UID: \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\") " pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:37:56.585638 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.585570 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xcns\" (UniqueName: \"kubernetes.io/projected/9de13670-914f-4618-a41f-f9c2124f33c2-kube-api-access-8xcns\") pod \"payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6\" (UID: \"9de13670-914f-4618-a41f-f9c2124f33c2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:37:56.585638 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.585620 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9de13670-914f-4618-a41f-f9c2124f33c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6\" (UID: \"9de13670-914f-4618-a41f-f9c2124f33c2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:37:56.586293 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.586275 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9de13670-914f-4618-a41f-f9c2124f33c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6\" (UID: \"9de13670-914f-4618-a41f-f9c2124f33c2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:37:56.607818 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.607782 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xcns\" (UniqueName: \"kubernetes.io/projected/9de13670-914f-4618-a41f-f9c2124f33c2-kube-api-access-8xcns\") pod \"payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6\" (UID: \"9de13670-914f-4618-a41f-f9c2124f33c2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:37:56.687092 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.686988 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n94zz\" (UniqueName: \"kubernetes.io/projected/1b36ca06-3d37-4ecb-8ac8-efde060d3115-kube-api-access-n94zz\") pod \"payload-processing-e2e-mt-c2a82a29-577d456474-hbc68\" (UID: \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\") " pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:37:56.687303 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.687143 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b36ca06-3d37-4ecb-8ac8-efde060d3115-plugins-config-volume\") pod \"payload-processing-e2e-mt-c2a82a29-577d456474-hbc68\" (UID: \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\") " pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:37:56.687842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.687820 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b36ca06-3d37-4ecb-8ac8-efde060d3115-plugins-config-volume\") pod \"payload-processing-e2e-mt-c2a82a29-577d456474-hbc68\" (UID: \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\") " pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:37:56.698812 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.698783 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n94zz\" (UniqueName: \"kubernetes.io/projected/1b36ca06-3d37-4ecb-8ac8-efde060d3115-kube-api-access-n94zz\") pod \"payload-processing-e2e-mt-c2a82a29-577d456474-hbc68\" (UID: \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\") " pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:37:56.739800 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.739763 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:37:56.832058 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.832028 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:37:56.977975 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:56.977951 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68"] Aug 04 16:37:56.980544 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:37:56.980514 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b36ca06_3d37_4ecb_8ac8_efde060d3115.slice/crio-ffaf117f9c2e22a1db9c9cf4f1ba43769f3a94419246b2fda72a4ae1075e613a WatchSource:0}: Error finding container ffaf117f9c2e22a1db9c9cf4f1ba43769f3a94419246b2fda72a4ae1075e613a: Status 404 returned error can't find the container with id ffaf117f9c2e22a1db9c9cf4f1ba43769f3a94419246b2fda72a4ae1075e613a Aug 04 16:37:57.092582 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:57.092540 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6"] Aug 04 16:37:57.094131 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:37:57.094105 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de13670_914f_4618_a41f_f9c2124f33c2.slice/crio-0c2d7d8c946c3e638aae114b88bac84ecafc6395809676f1dfa767d128396b28 WatchSource:0}: Error finding container 0c2d7d8c946c3e638aae114b88bac84ecafc6395809676f1dfa767d128396b28: Status 404 returned error can't find the container with id 0c2d7d8c946c3e638aae114b88bac84ecafc6395809676f1dfa767d128396b28 Aug 04 16:37:57.420500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:57.420457 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" event={"ID":"1b36ca06-3d37-4ecb-8ac8-efde060d3115","Type":"ContainerStarted","Data":"5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b"} Aug 04 16:37:57.420717 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:57.420667 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:37:57.420717 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:57.420705 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" event={"ID":"1b36ca06-3d37-4ecb-8ac8-efde060d3115","Type":"ContainerStarted","Data":"ffaf117f9c2e22a1db9c9cf4f1ba43769f3a94419246b2fda72a4ae1075e613a"} Aug 04 16:37:57.422181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:57.422153 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" event={"ID":"9de13670-914f-4618-a41f-f9c2124f33c2","Type":"ContainerStarted","Data":"e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb"} Aug 04 16:37:57.422292 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:57.422205 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" event={"ID":"9de13670-914f-4618-a41f-f9c2124f33c2","Type":"ContainerStarted","Data":"0c2d7d8c946c3e638aae114b88bac84ecafc6395809676f1dfa767d128396b28"} Aug 04 16:37:57.422292 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:57.422234 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:37:57.443305 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:57.443252 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" podStartSLOduration=1.443236933 podStartE2EDuration="1.443236933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:37:57.440860317 +0000 UTC m=+2506.763283798" watchObservedRunningTime="2026-08-04 16:37:57.443236933 +0000 UTC m=+2506.765660414" Aug 04 16:37:57.461644 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:37:57.461576 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" podStartSLOduration=1.461559073 podStartE2EDuration="1.461559073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:37:57.460845582 +0000 UTC m=+2506.783269062" watchObservedRunningTime="2026-08-04 16:37:57.461559073 +0000 UTC m=+2506.783982555" Aug 04 16:38:08.393648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.393548 2556 patch_prober.go:28] interesting pod/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 16:38:08.393648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.393620 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 16:38:08.429337 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.429307 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:38:08.429483 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.429386 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:38:08.486026 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.485964 2556 generic.go:356] "Generic (PLEG): container finished" podID="7c83c411-c019-439c-a89f-79ff6fe41186" containerID="37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44" exitCode=0 Aug 04 16:38:08.486026 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.486019 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" event={"ID":"7c83c411-c019-439c-a89f-79ff6fe41186","Type":"ContainerDied","Data":"37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44"} Aug 04 16:38:08.486529 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.486506 2556 scope.go:117] "RemoveContainer" containerID="37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44" Aug 04 16:38:08.487818 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.487770 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:38:08.487912 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.487883 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:38:08.487957 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:08.487932 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:38:09.496555 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:09.496513 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" event={"ID":"7c83c411-c019-439c-a89f-79ff6fe41186","Type":"ContainerStarted","Data":"37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158"} Aug 04 16:38:09.677062 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:09.677019 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:38:09.677062 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:09.677054 2556 patch_prober.go:28] interesting pod/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 16:38:09.677341 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:09.677120 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 16:38:10.677354 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:10.676893 2556 patch_prober.go:28] interesting pod/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 16:38:10.677354 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:10.676964 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 16:38:11.676688 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:11.676645 2556 patch_prober.go:28] interesting pod/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 16:38:11.676934 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:11.676715 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 16:38:12.677090 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:12.676615 2556 patch_prober.go:28] interesting pod/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 16:38:12.677090 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:12.676678 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 16:38:13.676799 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:13.676755 2556 patch_prober.go:28] interesting pod/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 16:38:13.677024 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:13.676838 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 16:38:14.681098 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:14.681065 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:38:14.681500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:14.681340 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:38:14.682076 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:14.682057 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:38:59.086500 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.086461 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt"] Aug 04 16:38:59.090316 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.090298 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" Aug 04 16:38:59.092998 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.092973 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-psz5k\"" Aug 04 16:38:59.098998 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.098972 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt"] Aug 04 16:38:59.102631 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.102602 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30381970-bdab-4a98-93fd-477ab09d5d43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt\" (UID: \"30381970-bdab-4a98-93fd-477ab09d5d43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" Aug 04 16:38:59.203177 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.203136 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30381970-bdab-4a98-93fd-477ab09d5d43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt\" (UID: \"30381970-bdab-4a98-93fd-477ab09d5d43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" Aug 04 16:38:59.203908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.203887 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30381970-bdab-4a98-93fd-477ab09d5d43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt\" (UID: \"30381970-bdab-4a98-93fd-477ab09d5d43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" Aug 04 16:38:59.402802 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.402708 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" Aug 04 16:38:59.544649 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.544621 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt"] Aug 04 16:38:59.548322 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:38:59.548295 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30381970_bdab_4a98_93fd_477ab09d5d43.slice/crio-4d69e69170122f7ae0b5be0f0e2b33cf87b1ed5a42544cd977a7d8715bc20f29 WatchSource:0}: Error finding container 4d69e69170122f7ae0b5be0f0e2b33cf87b1ed5a42544cd977a7d8715bc20f29: Status 404 returned error can't find the container with id 4d69e69170122f7ae0b5be0f0e2b33cf87b1ed5a42544cd977a7d8715bc20f29 Aug 04 16:38:59.772892 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:38:59.772850 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" event={"ID":"30381970-bdab-4a98-93fd-477ab09d5d43","Type":"ContainerStarted","Data":"4d69e69170122f7ae0b5be0f0e2b33cf87b1ed5a42544cd977a7d8715bc20f29"} Aug 04 16:39:00.780210 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:00.780173 2556 generic.go:356] "Generic (PLEG): container finished" podID="30381970-bdab-4a98-93fd-477ab09d5d43" containerID="622d4daa692bb9e8e06a866473f8fddf831eb633aee53ce41e90bc42ce3bbe00" exitCode=0 Aug 04 16:39:00.780594 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:00.780215 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" event={"ID":"30381970-bdab-4a98-93fd-477ab09d5d43","Type":"ContainerDied","Data":"622d4daa692bb9e8e06a866473f8fddf831eb633aee53ce41e90bc42ce3bbe00"} Aug 04 16:39:01.932634 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:01.932610 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" Aug 04 16:39:02.030303 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:02.030268 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30381970-bdab-4a98-93fd-477ab09d5d43-maas-api-service-ca\") pod \"30381970-bdab-4a98-93fd-477ab09d5d43\" (UID: \"30381970-bdab-4a98-93fd-477ab09d5d43\") " Aug 04 16:39:02.030672 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:02.030645 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30381970-bdab-4a98-93fd-477ab09d5d43-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "30381970-bdab-4a98-93fd-477ab09d5d43" (UID: "30381970-bdab-4a98-93fd-477ab09d5d43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:39:02.132112 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:02.132025 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30381970-bdab-4a98-93fd-477ab09d5d43-maas-api-service-ca\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:39:02.798272 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:02.798227 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" event={"ID":"30381970-bdab-4a98-93fd-477ab09d5d43","Type":"ContainerDied","Data":"4d69e69170122f7ae0b5be0f0e2b33cf87b1ed5a42544cd977a7d8715bc20f29"} Aug 04 16:39:02.798422 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:02.798290 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d69e69170122f7ae0b5be0f0e2b33cf87b1ed5a42544cd977a7d8715bc20f29" Aug 04 16:39:02.798422 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:02.798254 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt" Aug 04 16:39:14.889383 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:14.889343 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68"] Aug 04 16:39:14.889909 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:14.889625 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" podUID="1b36ca06-3d37-4ecb-8ac8-efde060d3115" containerName="payload-processing" containerID="cri-o://5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b" gracePeriod=30 Aug 04 16:39:14.898768 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:14.898739 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6"] Aug 04 16:39:14.899788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:14.899744 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" podUID="9de13670-914f-4618-a41f-f9c2124f33c2" containerName="payload-pre-processing" containerID="cri-o://e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb" gracePeriod=30 Aug 04 16:39:15.055272 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.055241 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt"] Aug 04 16:39:15.062446 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.062338 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2a82a29-9jhjt"] Aug 04 16:39:15.163360 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.163333 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:39:15.191496 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.191471 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:39:15.264889 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.264849 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n94zz\" (UniqueName: \"kubernetes.io/projected/1b36ca06-3d37-4ecb-8ac8-efde060d3115-kube-api-access-n94zz\") pod \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\" (UID: \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\") " Aug 04 16:39:15.265065 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.264903 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9de13670-914f-4618-a41f-f9c2124f33c2-plugins-config-volume\") pod \"9de13670-914f-4618-a41f-f9c2124f33c2\" (UID: \"9de13670-914f-4618-a41f-f9c2124f33c2\") " Aug 04 16:39:15.265065 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.264944 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b36ca06-3d37-4ecb-8ac8-efde060d3115-plugins-config-volume\") pod \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\" (UID: \"1b36ca06-3d37-4ecb-8ac8-efde060d3115\") " Aug 04 16:39:15.265065 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.264972 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xcns\" (UniqueName: \"kubernetes.io/projected/9de13670-914f-4618-a41f-f9c2124f33c2-kube-api-access-8xcns\") pod \"9de13670-914f-4618-a41f-f9c2124f33c2\" (UID: \"9de13670-914f-4618-a41f-f9c2124f33c2\") " Aug 04 16:39:15.265387 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.265354 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b36ca06-3d37-4ecb-8ac8-efde060d3115-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b36ca06-3d37-4ecb-8ac8-efde060d3115" (UID: "1b36ca06-3d37-4ecb-8ac8-efde060d3115"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:39:15.265387 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.265371 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9de13670-914f-4618-a41f-f9c2124f33c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9de13670-914f-4618-a41f-f9c2124f33c2" (UID: "9de13670-914f-4618-a41f-f9c2124f33c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:39:15.267150 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.267115 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9de13670-914f-4618-a41f-f9c2124f33c2-kube-api-access-8xcns" (OuterVolumeSpecName: "kube-api-access-8xcns") pod "9de13670-914f-4618-a41f-f9c2124f33c2" (UID: "9de13670-914f-4618-a41f-f9c2124f33c2"). InnerVolumeSpecName "kube-api-access-8xcns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:39:15.267150 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.267125 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b36ca06-3d37-4ecb-8ac8-efde060d3115-kube-api-access-n94zz" (OuterVolumeSpecName: "kube-api-access-n94zz") pod "1b36ca06-3d37-4ecb-8ac8-efde060d3115" (UID: "1b36ca06-3d37-4ecb-8ac8-efde060d3115"). InnerVolumeSpecName "kube-api-access-n94zz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:39:15.273843 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.273816 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30381970-bdab-4a98-93fd-477ab09d5d43" path="/var/lib/kubelet/pods/30381970-bdab-4a98-93fd-477ab09d5d43/volumes" Aug 04 16:39:15.366350 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.366308 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n94zz\" (UniqueName: \"kubernetes.io/projected/1b36ca06-3d37-4ecb-8ac8-efde060d3115-kube-api-access-n94zz\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:39:15.366350 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.366343 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9de13670-914f-4618-a41f-f9c2124f33c2-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:39:15.366350 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.366353 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b36ca06-3d37-4ecb-8ac8-efde060d3115-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:39:15.366350 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.366362 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xcns\" (UniqueName: \"kubernetes.io/projected/9de13670-914f-4618-a41f-f9c2124f33c2-kube-api-access-8xcns\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:39:15.867664 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.867626 2556 generic.go:356] "Generic (PLEG): container finished" podID="9de13670-914f-4618-a41f-f9c2124f33c2" containerID="e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb" exitCode=0 Aug 04 16:39:15.867868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.867672 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" Aug 04 16:39:15.867868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.867713 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" event={"ID":"9de13670-914f-4618-a41f-f9c2124f33c2","Type":"ContainerDied","Data":"e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb"} Aug 04 16:39:15.867868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.867800 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6" event={"ID":"9de13670-914f-4618-a41f-f9c2124f33c2","Type":"ContainerDied","Data":"0c2d7d8c946c3e638aae114b88bac84ecafc6395809676f1dfa767d128396b28"} Aug 04 16:39:15.867868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.867826 2556 scope.go:117] "RemoveContainer" containerID="e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb" Aug 04 16:39:15.869159 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.869132 2556 generic.go:356] "Generic (PLEG): container finished" podID="1b36ca06-3d37-4ecb-8ac8-efde060d3115" containerID="5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b" exitCode=0 Aug 04 16:39:15.869260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.869172 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" Aug 04 16:39:15.869260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.869168 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" event={"ID":"1b36ca06-3d37-4ecb-8ac8-efde060d3115","Type":"ContainerDied","Data":"5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b"} Aug 04 16:39:15.869372 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.869308 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68" event={"ID":"1b36ca06-3d37-4ecb-8ac8-efde060d3115","Type":"ContainerDied","Data":"ffaf117f9c2e22a1db9c9cf4f1ba43769f3a94419246b2fda72a4ae1075e613a"} Aug 04 16:39:15.876800 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.876780 2556 scope.go:117] "RemoveContainer" containerID="e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb" Aug 04 16:39:15.877092 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:39:15.877066 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb\": container with ID starting with e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb not found: ID does not exist" containerID="e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb" Aug 04 16:39:15.877177 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.877103 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb"} err="failed to get container status \"e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb\": rpc error: code = NotFound desc = could not find container \"e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb\": container with ID starting with e714debcdab3362c8da94f0c95ef82235c8a21e6e398954b4a01da970792ebdb not found: ID does not exist" Aug 04 16:39:15.877177 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.877125 2556 scope.go:117] "RemoveContainer" containerID="5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b" Aug 04 16:39:15.884915 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.884896 2556 scope.go:117] "RemoveContainer" containerID="5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b" Aug 04 16:39:15.885178 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:39:15.885159 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b\": container with ID starting with 5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b not found: ID does not exist" containerID="5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b" Aug 04 16:39:15.885238 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.885188 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b"} err="failed to get container status \"5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b\": rpc error: code = NotFound desc = could not find container \"5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b\": container with ID starting with 5f28870aadf14fd4f851eeb59138724be89e5f408c36848bfc92fd15cc86cb2b not found: ID does not exist" Aug 04 16:39:15.888541 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.888516 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68"] Aug 04 16:39:15.893404 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.893378 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c2a82a29-577d456474-hbc68"] Aug 04 16:39:15.903035 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.903010 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6"] Aug 04 16:39:15.910477 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:15.910451 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c2a82a29-6f6896d79b-ws6f6"] Aug 04 16:39:17.273880 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:17.273844 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b36ca06-3d37-4ecb-8ac8-efde060d3115" path="/var/lib/kubelet/pods/1b36ca06-3d37-4ecb-8ac8-efde060d3115/volumes" Aug 04 16:39:17.274265 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:39:17.274246 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9de13670-914f-4618-a41f-f9c2124f33c2" path="/var/lib/kubelet/pods/9de13670-914f-4618-a41f-f9c2124f33c2/volumes" Aug 04 16:40:06.791870 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:06.791833 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz"] Aug 04 16:40:06.792424 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:06.792091 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" containerID="cri-o://37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158" gracePeriod=30 Aug 04 16:40:12.049316 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.049290 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:40:12.168417 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.168319 2556 generic.go:356] "Generic (PLEG): container finished" podID="7c83c411-c019-439c-a89f-79ff6fe41186" containerID="37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158" exitCode=0 Aug 04 16:40:12.168417 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.168382 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" Aug 04 16:40:12.168417 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.168404 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" event={"ID":"7c83c411-c019-439c-a89f-79ff6fe41186","Type":"ContainerDied","Data":"37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158"} Aug 04 16:40:12.168695 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.168473 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz" event={"ID":"7c83c411-c019-439c-a89f-79ff6fe41186","Type":"ContainerDied","Data":"802d1235dc1bacd501fe7c2cccf547581bf9c61bd4c33103b6dd88d1223cb947"} Aug 04 16:40:12.168695 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.168507 2556 scope.go:117] "RemoveContainer" containerID="37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158" Aug 04 16:40:12.177307 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.177287 2556 scope.go:117] "RemoveContainer" containerID="37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44" Aug 04 16:40:12.185275 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.185255 2556 scope.go:117] "RemoveContainer" containerID="37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158" Aug 04 16:40:12.185566 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:40:12.185543 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158\": container with ID starting with 37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158 not found: ID does not exist" containerID="37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158" Aug 04 16:40:12.185619 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.185577 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158"} err="failed to get container status \"37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158\": rpc error: code = NotFound desc = could not find container \"37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158\": container with ID starting with 37575ce4e2cd694204dfb901165cae1a53b108a22d181fa60d3ef3097d7ad158 not found: ID does not exist" Aug 04 16:40:12.185619 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.185596 2556 scope.go:117] "RemoveContainer" containerID="37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44" Aug 04 16:40:12.185827 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:40:12.185808 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44\": container with ID starting with 37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44 not found: ID does not exist" containerID="37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44" Aug 04 16:40:12.185900 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.185848 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44"} err="failed to get container status \"37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44\": rpc error: code = NotFound desc = could not find container \"37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44\": container with ID starting with 37c1a4c3419754052ec82565ae20984a5d57c928790d3624e4c8c6340fc4ec44 not found: ID does not exist" Aug 04 16:40:12.208164 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208135 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwpxk\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-kube-api-access-zwpxk\") pod \"7c83c411-c019-439c-a89f-79ff6fe41186\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " Aug 04 16:40:12.208293 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208179 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-istio-token\") pod \"7c83c411-c019-439c-a89f-79ff6fe41186\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " Aug 04 16:40:12.208293 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208212 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-credential-socket\") pod \"7c83c411-c019-439c-a89f-79ff6fe41186\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " Aug 04 16:40:12.208293 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208255 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c83c411-c019-439c-a89f-79ff6fe41186-istiod-ca-cert\") pod \"7c83c411-c019-439c-a89f-79ff6fe41186\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " Aug 04 16:40:12.208466 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208309 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-certs\") pod \"7c83c411-c019-439c-a89f-79ff6fe41186\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " Aug 04 16:40:12.208466 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208348 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-data\") pod \"7c83c411-c019-439c-a89f-79ff6fe41186\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " Aug 04 16:40:12.208466 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208387 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-socket\") pod \"7c83c411-c019-439c-a89f-79ff6fe41186\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " Aug 04 16:40:12.208466 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208415 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c83c411-c019-439c-a89f-79ff6fe41186-istio-podinfo\") pod \"7c83c411-c019-439c-a89f-79ff6fe41186\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " Aug 04 16:40:12.208672 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208485 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-envoy\") pod \"7c83c411-c019-439c-a89f-79ff6fe41186\" (UID: \"7c83c411-c019-439c-a89f-79ff6fe41186\") " Aug 04 16:40:12.208672 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208566 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c83c411-c019-439c-a89f-79ff6fe41186" (UID: "7c83c411-c019-439c-a89f-79ff6fe41186"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:40:12.208672 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208589 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c83c411-c019-439c-a89f-79ff6fe41186" (UID: "7c83c411-c019-439c-a89f-79ff6fe41186"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:40:12.208827 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208666 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c83c411-c019-439c-a89f-79ff6fe41186-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c83c411-c019-439c-a89f-79ff6fe41186" (UID: "7c83c411-c019-439c-a89f-79ff6fe41186"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:40:12.208827 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208668 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c83c411-c019-439c-a89f-79ff6fe41186" (UID: "7c83c411-c019-439c-a89f-79ff6fe41186"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:40:12.208936 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208914 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:40:12.208972 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208937 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:40:12.208972 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208951 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:40:12.208972 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208949 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c83c411-c019-439c-a89f-79ff6fe41186" (UID: "7c83c411-c019-439c-a89f-79ff6fe41186"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:40:12.208972 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.208966 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c83c411-c019-439c-a89f-79ff6fe41186-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:40:12.210670 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.210648 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c83c411-c019-439c-a89f-79ff6fe41186" (UID: "7c83c411-c019-439c-a89f-79ff6fe41186"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:40:12.210800 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.210701 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c83c411-c019-439c-a89f-79ff6fe41186" (UID: "7c83c411-c019-439c-a89f-79ff6fe41186"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:40:12.210860 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.210794 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c83c411-c019-439c-a89f-79ff6fe41186-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c83c411-c019-439c-a89f-79ff6fe41186" (UID: "7c83c411-c019-439c-a89f-79ff6fe41186"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:40:12.210860 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.210838 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-kube-api-access-zwpxk" (OuterVolumeSpecName: "kube-api-access-zwpxk") pod "7c83c411-c019-439c-a89f-79ff6fe41186" (UID: "7c83c411-c019-439c-a89f-79ff6fe41186"). InnerVolumeSpecName "kube-api-access-zwpxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:40:12.309772 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.309717 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:40:12.309772 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.309769 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwpxk\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-kube-api-access-zwpxk\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:40:12.309772 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.309779 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c83c411-c019-439c-a89f-79ff6fe41186-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:40:12.310082 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.309789 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c83c411-c019-439c-a89f-79ff6fe41186-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:40:12.310082 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.309797 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c83c411-c019-439c-a89f-79ff6fe41186-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:40:12.492008 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.491979 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz"] Aug 04 16:40:12.498591 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:12.498559 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c2a82a29-openshift-default-cc7dd498d-265hz"] Aug 04 16:40:13.273072 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:40:13.273037 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" path="/var/lib/kubelet/pods/7c83c411-c019-439c-a89f-79ff6fe41186/volumes" Aug 04 16:41:14.809241 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809203 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7"] Aug 04 16:41:14.809649 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809636 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" Aug 04 16:41:14.809694 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809651 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" Aug 04 16:41:14.809694 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809676 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b36ca06-3d37-4ecb-8ac8-efde060d3115" containerName="payload-processing" Aug 04 16:41:14.809694 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809682 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b36ca06-3d37-4ecb-8ac8-efde060d3115" containerName="payload-processing" Aug 04 16:41:14.809814 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809693 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9de13670-914f-4618-a41f-f9c2124f33c2" containerName="payload-pre-processing" Aug 04 16:41:14.809814 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809700 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9de13670-914f-4618-a41f-f9c2124f33c2" containerName="payload-pre-processing" Aug 04 16:41:14.809814 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809709 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30381970-bdab-4a98-93fd-477ab09d5d43" containerName="revoke-keys" Aug 04 16:41:14.809814 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809714 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="30381970-bdab-4a98-93fd-477ab09d5d43" containerName="revoke-keys" Aug 04 16:41:14.809814 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809796 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b36ca06-3d37-4ecb-8ac8-efde060d3115" containerName="payload-processing" Aug 04 16:41:14.809814 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809805 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9de13670-914f-4618-a41f-f9c2124f33c2" containerName="payload-pre-processing" Aug 04 16:41:14.809997 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809816 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" Aug 04 16:41:14.809997 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.809825 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="30381970-bdab-4a98-93fd-477ab09d5d43" containerName="revoke-keys" Aug 04 16:41:14.814411 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.814394 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:14.817592 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.817567 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-df4f93-openshift-default-dockercfg-42ptb\"" Aug 04 16:41:14.830047 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.830022 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7"] Aug 04 16:41:14.898226 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.898190 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-socket\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:14.898438 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.898248 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-envoy\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:14.898438 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.898306 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-credential-socket\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:14.898438 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.898368 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-data\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:14.898438 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.898391 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-podinfo\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:14.898438 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.898435 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-certs\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:14.898752 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.898460 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-token\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:14.898752 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.898518 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istiod-ca-cert\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:14.898752 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:14.898546 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqs5c\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-kube-api-access-mqs5c\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000108 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000068 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-socket\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000108 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000116 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-envoy\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000374 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000147 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-credential-socket\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000374 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000184 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-data\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000374 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000201 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-podinfo\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000374 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000229 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-certs\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000374 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000256 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-token\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000374 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000304 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istiod-ca-cert\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000686 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000458 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqs5c\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-kube-api-access-mqs5c\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000686 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000604 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-data\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000827 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000715 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-socket\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000827 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000792 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-certs\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.000983 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.000964 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-credential-socket\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.001199 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.001180 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istiod-ca-cert\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.002608 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.002590 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-envoy\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.002768 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.002746 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-podinfo\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.011695 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.011670 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-token\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.011920 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.011899 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqs5c\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-kube-api-access-mqs5c\") pod \"e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.125338 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.125237 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:15.265263 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.265237 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7"] Aug 04 16:41:15.266817 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:41:15.266780 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd8f7d0_1333_4633_b6c8_1815227a2f3c.slice/crio-f57aeee55b4b1a8f55cb52f285088c91144b325cb646f41114b5caa1585d58f2 WatchSource:0}: Error finding container f57aeee55b4b1a8f55cb52f285088c91144b325cb646f41114b5caa1585d58f2: Status 404 returned error can't find the container with id f57aeee55b4b1a8f55cb52f285088c91144b325cb646f41114b5caa1585d58f2 Aug 04 16:41:15.269059 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.269021 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:41:15.269145 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.269108 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:41:15.269189 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.269149 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:41:15.494758 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.494705 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" event={"ID":"8bd8f7d0-1333-4633-b6c8-1815227a2f3c","Type":"ContainerStarted","Data":"04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14"} Aug 04 16:41:15.494982 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.494780 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" event={"ID":"8bd8f7d0-1333-4633-b6c8-1815227a2f3c","Type":"ContainerStarted","Data":"f57aeee55b4b1a8f55cb52f285088c91144b325cb646f41114b5caa1585d58f2"} Aug 04 16:41:15.534360 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:15.534304 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" podStartSLOduration=1.534287899 podStartE2EDuration="1.534287899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:41:15.532974633 +0000 UTC m=+2704.855398112" watchObservedRunningTime="2026-08-04 16:41:15.534287899 +0000 UTC m=+2704.856711378" Aug 04 16:41:16.125495 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:16.125455 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:16.130607 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:16.130583 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:16.499659 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:16.499631 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:16.500855 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:16.500823 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:19.635203 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.635164 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5"] Aug 04 16:41:19.635709 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.635690 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" Aug 04 16:41:19.635709 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.635711 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" Aug 04 16:41:19.635891 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.635864 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c83c411-c019-439c-a89f-79ff6fe41186" containerName="istio-proxy" Aug 04 16:41:19.639137 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.639116 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l"] Aug 04 16:41:19.639273 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.639251 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:19.642046 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.642023 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-df4f93\"" Aug 04 16:41:19.642531 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.642509 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:19.643010 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.642994 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-df4f93-dockercfg-kr6fm\"" Aug 04 16:41:19.653670 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.653647 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5"] Aug 04 16:41:19.673368 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.673335 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l"] Aug 04 16:41:19.745563 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.745520 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72732216-8b0c-4aee-8ad1-e97165365624-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l\" (UID: \"72732216-8b0c-4aee-8ad1-e97165365624\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:19.745774 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.745630 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc901b5-5983-42ec-be9b-bd4f9596534f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5\" (UID: \"2fc901b5-5983-42ec-be9b-bd4f9596534f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:19.745774 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.745666 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9twpf\" (UniqueName: \"kubernetes.io/projected/72732216-8b0c-4aee-8ad1-e97165365624-kube-api-access-9twpf\") pod \"payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l\" (UID: \"72732216-8b0c-4aee-8ad1-e97165365624\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:19.745774 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.745710 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9w57\" (UniqueName: \"kubernetes.io/projected/2fc901b5-5983-42ec-be9b-bd4f9596534f-kube-api-access-q9w57\") pod \"payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5\" (UID: \"2fc901b5-5983-42ec-be9b-bd4f9596534f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:19.846641 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.846601 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9w57\" (UniqueName: \"kubernetes.io/projected/2fc901b5-5983-42ec-be9b-bd4f9596534f-kube-api-access-q9w57\") pod \"payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5\" (UID: \"2fc901b5-5983-42ec-be9b-bd4f9596534f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:19.846835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.846691 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72732216-8b0c-4aee-8ad1-e97165365624-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l\" (UID: \"72732216-8b0c-4aee-8ad1-e97165365624\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:19.846835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.846760 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc901b5-5983-42ec-be9b-bd4f9596534f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5\" (UID: \"2fc901b5-5983-42ec-be9b-bd4f9596534f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:19.846835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.846784 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9twpf\" (UniqueName: \"kubernetes.io/projected/72732216-8b0c-4aee-8ad1-e97165365624-kube-api-access-9twpf\") pod \"payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l\" (UID: \"72732216-8b0c-4aee-8ad1-e97165365624\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:19.847427 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.847403 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72732216-8b0c-4aee-8ad1-e97165365624-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l\" (UID: \"72732216-8b0c-4aee-8ad1-e97165365624\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:19.847484 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.847466 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc901b5-5983-42ec-be9b-bd4f9596534f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5\" (UID: \"2fc901b5-5983-42ec-be9b-bd4f9596534f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:19.857375 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.857342 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9twpf\" (UniqueName: \"kubernetes.io/projected/72732216-8b0c-4aee-8ad1-e97165365624-kube-api-access-9twpf\") pod \"payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l\" (UID: \"72732216-8b0c-4aee-8ad1-e97165365624\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:19.857514 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.857440 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9w57\" (UniqueName: \"kubernetes.io/projected/2fc901b5-5983-42ec-be9b-bd4f9596534f-kube-api-access-q9w57\") pod \"payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5\" (UID: \"2fc901b5-5983-42ec-be9b-bd4f9596534f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:19.952129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.952035 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:19.961828 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:19.961782 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:20.110880 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.110847 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5"] Aug 04 16:41:20.113071 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:41:20.113035 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc901b5_5983_42ec_be9b_bd4f9596534f.slice/crio-92d6d85f27710062bf8b8497dbbb215cfc48d25942bcc75312d8fe7d8b65ed9a WatchSource:0}: Error finding container 92d6d85f27710062bf8b8497dbbb215cfc48d25942bcc75312d8fe7d8b65ed9a: Status 404 returned error can't find the container with id 92d6d85f27710062bf8b8497dbbb215cfc48d25942bcc75312d8fe7d8b65ed9a Aug 04 16:41:20.150841 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.150814 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l"] Aug 04 16:41:20.153365 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:41:20.153333 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72732216_8b0c_4aee_8ad1_e97165365624.slice/crio-4c7e4a3830e66685638852d91cca7e74a060a6dcb756498ff975ccb3ee89d3e9 WatchSource:0}: Error finding container 4c7e4a3830e66685638852d91cca7e74a060a6dcb756498ff975ccb3ee89d3e9: Status 404 returned error can't find the container with id 4c7e4a3830e66685638852d91cca7e74a060a6dcb756498ff975ccb3ee89d3e9 Aug 04 16:41:20.521437 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.521399 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" event={"ID":"2fc901b5-5983-42ec-be9b-bd4f9596534f","Type":"ContainerStarted","Data":"77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79"} Aug 04 16:41:20.521670 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.521466 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" event={"ID":"2fc901b5-5983-42ec-be9b-bd4f9596534f","Type":"ContainerStarted","Data":"92d6d85f27710062bf8b8497dbbb215cfc48d25942bcc75312d8fe7d8b65ed9a"} Aug 04 16:41:20.521670 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.521499 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:20.523038 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.523011 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" event={"ID":"72732216-8b0c-4aee-8ad1-e97165365624","Type":"ContainerStarted","Data":"07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81"} Aug 04 16:41:20.523168 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.523054 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" event={"ID":"72732216-8b0c-4aee-8ad1-e97165365624","Type":"ContainerStarted","Data":"4c7e4a3830e66685638852d91cca7e74a060a6dcb756498ff975ccb3ee89d3e9"} Aug 04 16:41:20.523231 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.523165 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:20.541536 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.541482 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" podStartSLOduration=1.541464814 podStartE2EDuration="1.541464814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:41:20.538839355 +0000 UTC m=+2709.861262848" watchObservedRunningTime="2026-08-04 16:41:20.541464814 +0000 UTC m=+2709.863888293" Aug 04 16:41:20.556995 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:20.556943 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" podStartSLOduration=1.556926429 podStartE2EDuration="1.556926429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:41:20.555596773 +0000 UTC m=+2709.878020255" watchObservedRunningTime="2026-08-04 16:41:20.556926429 +0000 UTC m=+2709.879349936" Aug 04 16:41:31.530353 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:31.530317 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:41:31.530841 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:31.530386 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:41:31.835928 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:31.835150 2556 patch_prober.go:28] interesting pod/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": read tcp 10.132.0.2:42122->10.132.0.64:15021: read: connection reset by peer" start-of-body= Aug 04 16:41:31.835928 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:31.835230 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": read tcp 10.132.0.2:42122->10.132.0.64:15021: read: connection reset by peer" Aug 04 16:41:32.588808 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:32.588773 2556 generic.go:356] "Generic (PLEG): container finished" podID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerID="04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14" exitCode=0 Aug 04 16:41:32.589195 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:32.588836 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" event={"ID":"8bd8f7d0-1333-4633-b6c8-1815227a2f3c","Type":"ContainerDied","Data":"04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14"} Aug 04 16:41:32.589243 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:32.589205 2556 scope.go:117] "RemoveContainer" containerID="04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14" Aug 04 16:41:32.590237 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:32.590208 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:41:32.590347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:32.590293 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:41:32.590347 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:32.590339 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:41:33.602755 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:33.602704 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" event={"ID":"8bd8f7d0-1333-4633-b6c8-1815227a2f3c","Type":"ContainerStarted","Data":"26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2"} Aug 04 16:41:34.126411 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:34.126375 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:34.126748 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:34.126698 2556 patch_prober.go:28] interesting pod/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Aug 04 16:41:34.126882 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:34.126796 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Aug 04 16:41:35.126482 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:35.126437 2556 patch_prober.go:28] interesting pod/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Aug 04 16:41:35.126922 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:35.126510 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Aug 04 16:41:36.126145 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:36.126107 2556 patch_prober.go:28] interesting pod/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Aug 04 16:41:36.126330 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:36.126176 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Aug 04 16:41:37.126200 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:37.126155 2556 patch_prober.go:28] interesting pod/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Aug 04 16:41:37.126620 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:37.126218 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Aug 04 16:41:38.126592 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:38.126555 2556 patch_prober.go:28] interesting pod/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Aug 04 16:41:38.127025 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:38.126627 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Aug 04 16:41:39.129517 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:39.129482 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:39.129911 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:39.129759 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:39.130580 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:39.130565 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:41:43.198763 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.198717 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr"] Aug 04 16:41:43.202825 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.202800 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.209917 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.209892 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1f0f0c-openshift-default-dockercfg-p96cn\"" Aug 04 16:41:43.219629 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.219598 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr"] Aug 04 16:41:43.290830 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.290786 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-certs\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.291006 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.290931 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-data\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.291006 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.290966 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-envoy\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.291082 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.291005 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-podinfo\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.291082 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.291029 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-socket\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.291082 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.291051 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-credential-socket\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.291226 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.291118 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkv9m\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-kube-api-access-bkv9m\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.291226 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.291153 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-token\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.291226 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.291174 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istiod-ca-cert\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.391877 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.391834 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-data\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.391877 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.391882 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-envoy\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.391913 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-podinfo\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.391936 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-socket\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.391955 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-credential-socket\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.391977 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkv9m\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-kube-api-access-bkv9m\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.392002 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-token\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.392028 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istiod-ca-cert\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.392068 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-certs\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392422 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.392295 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-data\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392608 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.392585 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-certs\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392775 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.392584 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-credential-socket\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.392899 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.392666 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-socket\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.393145 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.393122 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istiod-ca-cert\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.394539 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.394516 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-envoy\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.394645 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.394622 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-podinfo\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.400692 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.400660 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-token\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.400809 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.400789 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkv9m\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-kube-api-access-bkv9m\") pod \"e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.513218 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.513180 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:43.659779 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.657997 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr"] Aug 04 16:41:43.662256 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.662208 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:41:43.662393 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.662360 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:41:43.662481 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:43.662458 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:41:44.662293 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:44.662250 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" event={"ID":"119da46a-a7b6-4261-ae7a-dd9f1d1c4117","Type":"ContainerStarted","Data":"b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6"} Aug 04 16:41:44.662791 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:44.662318 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" event={"ID":"119da46a-a7b6-4261-ae7a-dd9f1d1c4117","Type":"ContainerStarted","Data":"2b1489bc6dd9d346764c038d4db65c4729cc82822da9a0709a2d57cb4354f7e1"} Aug 04 16:41:44.687216 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:44.687158 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" podStartSLOduration=1.687142785 podStartE2EDuration="1.687142785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:41:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:41:44.68577015 +0000 UTC m=+2734.008193630" watchObservedRunningTime="2026-08-04 16:41:44.687142785 +0000 UTC m=+2734.009566265" Aug 04 16:41:45.514405 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:45.514369 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:45.519484 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:45.519453 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:45.667628 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:45.667594 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:45.668501 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:45.668482 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:41:48.666658 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.666625 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx"] Aug 04 16:41:48.670489 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.670472 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:41:48.673182 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.673162 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-1f0f0c-dockercfg-lstvh\"" Aug 04 16:41:48.673287 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.673213 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-1f0f0c\"" Aug 04 16:41:48.688181 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.688151 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx"] Aug 04 16:41:48.741019 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.740984 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k"] Aug 04 16:41:48.744867 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.744836 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:41:48.749532 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.749510 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzjbr\" (UniqueName: \"kubernetes.io/projected/91937e79-47a7-4bf0-986e-01bca13362b8-kube-api-access-gzjbr\") pod \"payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx\" (UID: \"91937e79-47a7-4bf0-986e-01bca13362b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:41:48.749652 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.749575 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91937e79-47a7-4bf0-986e-01bca13362b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx\" (UID: \"91937e79-47a7-4bf0-986e-01bca13362b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:41:48.769082 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.769053 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k"] Aug 04 16:41:48.851021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.850983 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6405d4d0-9add-444d-a3db-d9c17d9cd11e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k\" (UID: \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:41:48.851216 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.851069 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmgt6\" (UniqueName: \"kubernetes.io/projected/6405d4d0-9add-444d-a3db-d9c17d9cd11e-kube-api-access-rmgt6\") pod \"payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k\" (UID: \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:41:48.851216 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.851099 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzjbr\" (UniqueName: \"kubernetes.io/projected/91937e79-47a7-4bf0-986e-01bca13362b8-kube-api-access-gzjbr\") pod \"payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx\" (UID: \"91937e79-47a7-4bf0-986e-01bca13362b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:41:48.851326 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.851236 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91937e79-47a7-4bf0-986e-01bca13362b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx\" (UID: \"91937e79-47a7-4bf0-986e-01bca13362b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:41:48.851982 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.851961 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91937e79-47a7-4bf0-986e-01bca13362b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx\" (UID: \"91937e79-47a7-4bf0-986e-01bca13362b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:41:48.862547 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.862522 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzjbr\" (UniqueName: \"kubernetes.io/projected/91937e79-47a7-4bf0-986e-01bca13362b8-kube-api-access-gzjbr\") pod \"payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx\" (UID: \"91937e79-47a7-4bf0-986e-01bca13362b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:41:48.952021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.951935 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6405d4d0-9add-444d-a3db-d9c17d9cd11e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k\" (UID: \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:41:48.952021 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.952014 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmgt6\" (UniqueName: \"kubernetes.io/projected/6405d4d0-9add-444d-a3db-d9c17d9cd11e-kube-api-access-rmgt6\") pod \"payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k\" (UID: \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:41:48.952662 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.952641 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6405d4d0-9add-444d-a3db-d9c17d9cd11e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k\" (UID: \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:41:48.963644 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.963617 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmgt6\" (UniqueName: \"kubernetes.io/projected/6405d4d0-9add-444d-a3db-d9c17d9cd11e-kube-api-access-rmgt6\") pod \"payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k\" (UID: \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:41:48.982056 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:48.982027 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:41:49.055977 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.055943 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:41:49.137745 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.137703 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx"] Aug 04 16:41:49.140231 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:41:49.140203 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91937e79_47a7_4bf0_986e_01bca13362b8.slice/crio-b7b3410f8473b87d8dd306304c2f4a708ff0fa9bea07b97c04842069721bef82 WatchSource:0}: Error finding container b7b3410f8473b87d8dd306304c2f4a708ff0fa9bea07b97c04842069721bef82: Status 404 returned error can't find the container with id b7b3410f8473b87d8dd306304c2f4a708ff0fa9bea07b97c04842069721bef82 Aug 04 16:41:49.218777 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.218754 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k"] Aug 04 16:41:49.221558 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:41:49.221530 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6405d4d0_9add_444d_a3db_d9c17d9cd11e.slice/crio-7674ae804d4c10266409d0e1419bb047efa0285cb37d82ed89632352ed5cfe67 WatchSource:0}: Error finding container 7674ae804d4c10266409d0e1419bb047efa0285cb37d82ed89632352ed5cfe67: Status 404 returned error can't find the container with id 7674ae804d4c10266409d0e1419bb047efa0285cb37d82ed89632352ed5cfe67 Aug 04 16:41:49.692244 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.692204 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" event={"ID":"6405d4d0-9add-444d-a3db-d9c17d9cd11e","Type":"ContainerStarted","Data":"263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b"} Aug 04 16:41:49.692768 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.692278 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:41:49.692768 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.692306 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" event={"ID":"6405d4d0-9add-444d-a3db-d9c17d9cd11e","Type":"ContainerStarted","Data":"7674ae804d4c10266409d0e1419bb047efa0285cb37d82ed89632352ed5cfe67"} Aug 04 16:41:49.694131 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.694096 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" event={"ID":"91937e79-47a7-4bf0-986e-01bca13362b8","Type":"ContainerStarted","Data":"3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe"} Aug 04 16:41:49.694248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.694150 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" event={"ID":"91937e79-47a7-4bf0-986e-01bca13362b8","Type":"ContainerStarted","Data":"b7b3410f8473b87d8dd306304c2f4a708ff0fa9bea07b97c04842069721bef82"} Aug 04 16:41:49.694248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.694198 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:41:49.719205 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.719156 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" podStartSLOduration=1.71914104 podStartE2EDuration="1.71914104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:41:49.71654201 +0000 UTC m=+2739.038965501" watchObservedRunningTime="2026-08-04 16:41:49.71914104 +0000 UTC m=+2739.041564520" Aug 04 16:41:49.744619 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:41:49.744568 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" podStartSLOduration=1.7445538539999998 podStartE2EDuration="1.744553854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:41:49.743077759 +0000 UTC m=+2739.065501235" watchObservedRunningTime="2026-08-04 16:41:49.744553854 +0000 UTC m=+2739.066977366" Aug 04 16:42:00.669567 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.669530 2556 patch_prober.go:28] interesting pod/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Aug 04 16:42:00.670028 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.669587 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Aug 04 16:42:00.702146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.702115 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:42:00.702358 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.702342 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:42:00.761085 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.761052 2556 generic.go:356] "Generic (PLEG): container finished" podID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerID="b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6" exitCode=0 Aug 04 16:42:00.761247 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.761130 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" event={"ID":"119da46a-a7b6-4261-ae7a-dd9f1d1c4117","Type":"ContainerDied","Data":"b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6"} Aug 04 16:42:00.761703 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.761681 2556 scope.go:117] "RemoveContainer" containerID="b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6" Aug 04 16:42:00.763038 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.762974 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:42:00.763173 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.763095 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:42:00.763173 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:00.763148 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:42:01.768583 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:01.768549 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" event={"ID":"119da46a-a7b6-4261-ae7a-dd9f1d1c4117","Type":"ContainerStarted","Data":"c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d"} Aug 04 16:42:02.513485 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:02.513455 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:42:02.513797 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:02.513765 2556 patch_prober.go:28] interesting pod/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Aug 04 16:42:02.513959 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:02.513818 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Aug 04 16:42:03.514122 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:03.514078 2556 patch_prober.go:28] interesting pod/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Aug 04 16:42:03.514511 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:03.514136 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Aug 04 16:42:04.514050 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:04.514013 2556 patch_prober.go:28] interesting pod/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Aug 04 16:42:04.514228 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:04.514070 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Aug 04 16:42:05.514178 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:05.514140 2556 patch_prober.go:28] interesting pod/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Aug 04 16:42:05.514360 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:05.514214 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Aug 04 16:42:06.514205 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:06.514164 2556 patch_prober.go:28] interesting pod/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Aug 04 16:42:06.514391 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:06.514222 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Aug 04 16:42:07.517682 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:07.517655 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:42:07.518077 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:07.517926 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:42:07.518678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:42:07.518663 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:43:43.375450 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.375412 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5"] Aug 04 16:43:43.376052 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.375711 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" podUID="2fc901b5-5983-42ec-be9b-bd4f9596534f" containerName="payload-processing" containerID="cri-o://77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79" gracePeriod=30 Aug 04 16:43:43.383572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.383544 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l"] Aug 04 16:43:43.383880 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.383834 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" podUID="72732216-8b0c-4aee-8ad1-e97165365624" containerName="payload-pre-processing" containerID="cri-o://07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81" gracePeriod=30 Aug 04 16:43:43.674978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.674945 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:43:43.677875 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.677853 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:43:43.821655 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.821623 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9w57\" (UniqueName: \"kubernetes.io/projected/2fc901b5-5983-42ec-be9b-bd4f9596534f-kube-api-access-q9w57\") pod \"2fc901b5-5983-42ec-be9b-bd4f9596534f\" (UID: \"2fc901b5-5983-42ec-be9b-bd4f9596534f\") " Aug 04 16:43:43.821879 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.821682 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9twpf\" (UniqueName: \"kubernetes.io/projected/72732216-8b0c-4aee-8ad1-e97165365624-kube-api-access-9twpf\") pod \"72732216-8b0c-4aee-8ad1-e97165365624\" (UID: \"72732216-8b0c-4aee-8ad1-e97165365624\") " Aug 04 16:43:43.821879 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.821709 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72732216-8b0c-4aee-8ad1-e97165365624-plugins-config-volume\") pod \"72732216-8b0c-4aee-8ad1-e97165365624\" (UID: \"72732216-8b0c-4aee-8ad1-e97165365624\") " Aug 04 16:43:43.821879 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.821832 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc901b5-5983-42ec-be9b-bd4f9596534f-plugins-config-volume\") pod \"2fc901b5-5983-42ec-be9b-bd4f9596534f\" (UID: \"2fc901b5-5983-42ec-be9b-bd4f9596534f\") " Aug 04 16:43:43.822161 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.822132 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fc901b5-5983-42ec-be9b-bd4f9596534f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2fc901b5-5983-42ec-be9b-bd4f9596534f" (UID: "2fc901b5-5983-42ec-be9b-bd4f9596534f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:43:43.822248 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.822170 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72732216-8b0c-4aee-8ad1-e97165365624-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72732216-8b0c-4aee-8ad1-e97165365624" (UID: "72732216-8b0c-4aee-8ad1-e97165365624"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:43:43.823936 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.823911 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72732216-8b0c-4aee-8ad1-e97165365624-kube-api-access-9twpf" (OuterVolumeSpecName: "kube-api-access-9twpf") pod "72732216-8b0c-4aee-8ad1-e97165365624" (UID: "72732216-8b0c-4aee-8ad1-e97165365624"). InnerVolumeSpecName "kube-api-access-9twpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:43:43.823999 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.823965 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc901b5-5983-42ec-be9b-bd4f9596534f-kube-api-access-q9w57" (OuterVolumeSpecName: "kube-api-access-q9w57") pod "2fc901b5-5983-42ec-be9b-bd4f9596534f" (UID: "2fc901b5-5983-42ec-be9b-bd4f9596534f"). InnerVolumeSpecName "kube-api-access-q9w57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:43:43.922692 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.922613 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9twpf\" (UniqueName: \"kubernetes.io/projected/72732216-8b0c-4aee-8ad1-e97165365624-kube-api-access-9twpf\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:43.922692 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.922641 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72732216-8b0c-4aee-8ad1-e97165365624-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:43.922692 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.922650 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc901b5-5983-42ec-be9b-bd4f9596534f-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:43.922692 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:43.922659 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9w57\" (UniqueName: \"kubernetes.io/projected/2fc901b5-5983-42ec-be9b-bd4f9596534f-kube-api-access-q9w57\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:44.342194 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.342152 2556 generic.go:356] "Generic (PLEG): container finished" podID="72732216-8b0c-4aee-8ad1-e97165365624" containerID="07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81" exitCode=0 Aug 04 16:43:44.342380 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.342200 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" Aug 04 16:43:44.342380 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.342235 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" event={"ID":"72732216-8b0c-4aee-8ad1-e97165365624","Type":"ContainerDied","Data":"07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81"} Aug 04 16:43:44.342380 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.342297 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l" event={"ID":"72732216-8b0c-4aee-8ad1-e97165365624","Type":"ContainerDied","Data":"4c7e4a3830e66685638852d91cca7e74a060a6dcb756498ff975ccb3ee89d3e9"} Aug 04 16:43:44.342380 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.342321 2556 scope.go:117] "RemoveContainer" containerID="07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81" Aug 04 16:43:44.343747 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.343710 2556 generic.go:356] "Generic (PLEG): container finished" podID="2fc901b5-5983-42ec-be9b-bd4f9596534f" containerID="77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79" exitCode=0 Aug 04 16:43:44.343855 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.343756 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" event={"ID":"2fc901b5-5983-42ec-be9b-bd4f9596534f","Type":"ContainerDied","Data":"77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79"} Aug 04 16:43:44.343855 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.343809 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" event={"ID":"2fc901b5-5983-42ec-be9b-bd4f9596534f","Type":"ContainerDied","Data":"92d6d85f27710062bf8b8497dbbb215cfc48d25942bcc75312d8fe7d8b65ed9a"} Aug 04 16:43:44.343855 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.343811 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5" Aug 04 16:43:44.352869 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.352850 2556 scope.go:117] "RemoveContainer" containerID="07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81" Aug 04 16:43:44.353239 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:43:44.353217 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81\": container with ID starting with 07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81 not found: ID does not exist" containerID="07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81" Aug 04 16:43:44.353340 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.353249 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81"} err="failed to get container status \"07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81\": rpc error: code = NotFound desc = could not find container \"07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81\": container with ID starting with 07f9c38090c5459fee140268b3500861faac63e9c8a3bfff5d6e22e8b4a43f81 not found: ID does not exist" Aug 04 16:43:44.353340 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.353271 2556 scope.go:117] "RemoveContainer" containerID="77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79" Aug 04 16:43:44.363289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.363269 2556 scope.go:117] "RemoveContainer" containerID="77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79" Aug 04 16:43:44.363576 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:43:44.363555 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79\": container with ID starting with 77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79 not found: ID does not exist" containerID="77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79" Aug 04 16:43:44.363625 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.363585 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79"} err="failed to get container status \"77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79\": rpc error: code = NotFound desc = could not find container \"77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79\": container with ID starting with 77d31f3db77790c6df303a6b7f79dc08305b96fc755e0043e5869e23bd755c79 not found: ID does not exist" Aug 04 16:43:44.369991 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.369965 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l"] Aug 04 16:43:44.375839 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.375812 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df4f93-68985499b-lx76l"] Aug 04 16:43:44.387936 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.387904 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5"] Aug 04 16:43:44.396046 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:44.396007 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df4f93-7cb7ddc67d-8mjm5"] Aug 04 16:43:45.274364 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:45.274331 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fc901b5-5983-42ec-be9b-bd4f9596534f" path="/var/lib/kubelet/pods/2fc901b5-5983-42ec-be9b-bd4f9596534f/volumes" Aug 04 16:43:45.274716 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:45.274703 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72732216-8b0c-4aee-8ad1-e97165365624" path="/var/lib/kubelet/pods/72732216-8b0c-4aee-8ad1-e97165365624/volumes" Aug 04 16:43:50.049532 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:50.049497 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7"] Aug 04 16:43:50.050095 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:50.049820 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" containerID="cri-o://26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2" gracePeriod=30 Aug 04 16:43:54.132179 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:54.132138 2556 patch_prober.go:28] interesting pod/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 16:43:54.132574 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:54.132215 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:43:55.308853 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.308824 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:43:55.407718 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.407632 2556 generic.go:356] "Generic (PLEG): container finished" podID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerID="26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2" exitCode=0 Aug 04 16:43:55.407718 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.407681 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" Aug 04 16:43:55.407718 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.407697 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" event={"ID":"8bd8f7d0-1333-4633-b6c8-1815227a2f3c","Type":"ContainerDied","Data":"26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2"} Aug 04 16:43:55.407952 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.407758 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7" event={"ID":"8bd8f7d0-1333-4633-b6c8-1815227a2f3c","Type":"ContainerDied","Data":"f57aeee55b4b1a8f55cb52f285088c91144b325cb646f41114b5caa1585d58f2"} Aug 04 16:43:55.407952 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.407780 2556 scope.go:117] "RemoveContainer" containerID="26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2" Aug 04 16:43:55.417139 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.417119 2556 scope.go:117] "RemoveContainer" containerID="04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14" Aug 04 16:43:55.425316 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.425293 2556 scope.go:117] "RemoveContainer" containerID="26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2" Aug 04 16:43:55.425579 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:43:55.425560 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2\": container with ID starting with 26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2 not found: ID does not exist" containerID="26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2" Aug 04 16:43:55.425671 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.425589 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2"} err="failed to get container status \"26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2\": rpc error: code = NotFound desc = could not find container \"26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2\": container with ID starting with 26161c9526fb1cc4cf70dfc9605fa2fb3188324314f42bb7756559da6ab350c2 not found: ID does not exist" Aug 04 16:43:55.425671 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.425607 2556 scope.go:117] "RemoveContainer" containerID="04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14" Aug 04 16:43:55.425865 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:43:55.425847 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14\": container with ID starting with 04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14 not found: ID does not exist" containerID="04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14" Aug 04 16:43:55.425911 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.425872 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14"} err="failed to get container status \"04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14\": rpc error: code = NotFound desc = could not find container \"04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14\": container with ID starting with 04ee226484ddec805bc12fc5627c0c6f003c3b1acb2f22a7065bb094a69d6e14 not found: ID does not exist" Aug 04 16:43:55.439223 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439198 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-envoy\") pod \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " Aug 04 16:43:55.439368 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439260 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-token\") pod \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " Aug 04 16:43:55.439368 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439289 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-credential-socket\") pod \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " Aug 04 16:43:55.439368 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439314 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-certs\") pod \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " Aug 04 16:43:55.439368 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439336 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istiod-ca-cert\") pod \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " Aug 04 16:43:55.439584 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439405 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-podinfo\") pod \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " Aug 04 16:43:55.439584 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439438 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqs5c\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-kube-api-access-mqs5c\") pod \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " Aug 04 16:43:55.439584 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439494 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-socket\") pod \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " Aug 04 16:43:55.439584 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439541 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-data\") pod \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\" (UID: \"8bd8f7d0-1333-4633-b6c8-1815227a2f3c\") " Aug 04 16:43:55.439833 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439607 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8bd8f7d0-1333-4633-b6c8-1815227a2f3c" (UID: "8bd8f7d0-1333-4633-b6c8-1815227a2f3c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:43:55.439833 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439624 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8bd8f7d0-1333-4633-b6c8-1815227a2f3c" (UID: "8bd8f7d0-1333-4633-b6c8-1815227a2f3c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:43:55.439951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439882 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-data" (OuterVolumeSpecName: "istio-data") pod "8bd8f7d0-1333-4633-b6c8-1815227a2f3c" (UID: "8bd8f7d0-1333-4633-b6c8-1815227a2f3c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:43:55.439951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439809 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8bd8f7d0-1333-4633-b6c8-1815227a2f3c" (UID: "8bd8f7d0-1333-4633-b6c8-1815227a2f3c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:43:55.439951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439900 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8bd8f7d0-1333-4633-b6c8-1815227a2f3c" (UID: "8bd8f7d0-1333-4633-b6c8-1815227a2f3c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:43:55.439951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439929 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:55.439951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.439947 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:55.441556 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.441528 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8bd8f7d0-1333-4633-b6c8-1815227a2f3c" (UID: "8bd8f7d0-1333-4633-b6c8-1815227a2f3c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:43:55.441556 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.441546 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-token" (OuterVolumeSpecName: "istio-token") pod "8bd8f7d0-1333-4633-b6c8-1815227a2f3c" (UID: "8bd8f7d0-1333-4633-b6c8-1815227a2f3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:43:55.441692 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.441563 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-kube-api-access-mqs5c" (OuterVolumeSpecName: "kube-api-access-mqs5c") pod "8bd8f7d0-1333-4633-b6c8-1815227a2f3c" (UID: "8bd8f7d0-1333-4633-b6c8-1815227a2f3c"). InnerVolumeSpecName "kube-api-access-mqs5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:43:55.441692 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.441632 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8bd8f7d0-1333-4633-b6c8-1815227a2f3c" (UID: "8bd8f7d0-1333-4633-b6c8-1815227a2f3c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:43:55.541385 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.541347 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:55.541385 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.541379 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:55.541385 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.541392 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:55.541627 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.541401 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqs5c\" (UniqueName: \"kubernetes.io/projected/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-kube-api-access-mqs5c\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:55.541627 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.541410 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:55.541627 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.541419 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:55.541627 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.541427 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd8f7d0-1333-4633-b6c8-1815227a2f3c-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:43:55.731779 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.731746 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7"] Aug 04 16:43:55.733609 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:55.733586 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-df4f93-openshift-default-5f5876d69d-sb8h7"] Aug 04 16:43:57.273197 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:43:57.273162 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" path="/var/lib/kubelet/pods/8bd8f7d0-1333-4633-b6c8-1815227a2f3c/volumes" Aug 04 16:44:01.038643 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.038609 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k"] Aug 04 16:44:01.039199 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.038875 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" podUID="6405d4d0-9add-444d-a3db-d9c17d9cd11e" containerName="payload-processing" containerID="cri-o://263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b" gracePeriod=30 Aug 04 16:44:01.049330 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.047765 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx"] Aug 04 16:44:01.049330 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.047961 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" podUID="91937e79-47a7-4bf0-986e-01bca13362b8" containerName="payload-pre-processing" containerID="cri-o://3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe" gracePeriod=30 Aug 04 16:44:01.319813 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.319787 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:44:01.352074 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.352048 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:44:01.391943 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.391908 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmgt6\" (UniqueName: \"kubernetes.io/projected/6405d4d0-9add-444d-a3db-d9c17d9cd11e-kube-api-access-rmgt6\") pod \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\" (UID: \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\") " Aug 04 16:44:01.392129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.392015 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6405d4d0-9add-444d-a3db-d9c17d9cd11e-plugins-config-volume\") pod \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\" (UID: \"6405d4d0-9add-444d-a3db-d9c17d9cd11e\") " Aug 04 16:44:01.392129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.392067 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzjbr\" (UniqueName: \"kubernetes.io/projected/91937e79-47a7-4bf0-986e-01bca13362b8-kube-api-access-gzjbr\") pod \"91937e79-47a7-4bf0-986e-01bca13362b8\" (UID: \"91937e79-47a7-4bf0-986e-01bca13362b8\") " Aug 04 16:44:01.392129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.392095 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91937e79-47a7-4bf0-986e-01bca13362b8-plugins-config-volume\") pod \"91937e79-47a7-4bf0-986e-01bca13362b8\" (UID: \"91937e79-47a7-4bf0-986e-01bca13362b8\") " Aug 04 16:44:01.392392 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.392368 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6405d4d0-9add-444d-a3db-d9c17d9cd11e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6405d4d0-9add-444d-a3db-d9c17d9cd11e" (UID: "6405d4d0-9add-444d-a3db-d9c17d9cd11e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:44:01.392475 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.392463 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6405d4d0-9add-444d-a3db-d9c17d9cd11e-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:01.392716 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.392688 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91937e79-47a7-4bf0-986e-01bca13362b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "91937e79-47a7-4bf0-986e-01bca13362b8" (UID: "91937e79-47a7-4bf0-986e-01bca13362b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:44:01.394311 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.394277 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91937e79-47a7-4bf0-986e-01bca13362b8-kube-api-access-gzjbr" (OuterVolumeSpecName: "kube-api-access-gzjbr") pod "91937e79-47a7-4bf0-986e-01bca13362b8" (UID: "91937e79-47a7-4bf0-986e-01bca13362b8"). InnerVolumeSpecName "kube-api-access-gzjbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:44:01.394580 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.394549 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6405d4d0-9add-444d-a3db-d9c17d9cd11e-kube-api-access-rmgt6" (OuterVolumeSpecName: "kube-api-access-rmgt6") pod "6405d4d0-9add-444d-a3db-d9c17d9cd11e" (UID: "6405d4d0-9add-444d-a3db-d9c17d9cd11e"). InnerVolumeSpecName "kube-api-access-rmgt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:44:01.445790 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.445720 2556 generic.go:356] "Generic (PLEG): container finished" podID="91937e79-47a7-4bf0-986e-01bca13362b8" containerID="3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe" exitCode=0 Aug 04 16:44:01.445790 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.445797 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" Aug 04 16:44:01.446017 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.445835 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" event={"ID":"91937e79-47a7-4bf0-986e-01bca13362b8","Type":"ContainerDied","Data":"3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe"} Aug 04 16:44:01.446017 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.445912 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx" event={"ID":"91937e79-47a7-4bf0-986e-01bca13362b8","Type":"ContainerDied","Data":"b7b3410f8473b87d8dd306304c2f4a708ff0fa9bea07b97c04842069721bef82"} Aug 04 16:44:01.446017 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.445946 2556 scope.go:117] "RemoveContainer" containerID="3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe" Aug 04 16:44:01.447255 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.447222 2556 generic.go:356] "Generic (PLEG): container finished" podID="6405d4d0-9add-444d-a3db-d9c17d9cd11e" containerID="263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b" exitCode=0 Aug 04 16:44:01.447357 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.447252 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" event={"ID":"6405d4d0-9add-444d-a3db-d9c17d9cd11e","Type":"ContainerDied","Data":"263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b"} Aug 04 16:44:01.447357 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.447260 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" Aug 04 16:44:01.447357 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.447304 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k" event={"ID":"6405d4d0-9add-444d-a3db-d9c17d9cd11e","Type":"ContainerDied","Data":"7674ae804d4c10266409d0e1419bb047efa0285cb37d82ed89632352ed5cfe67"} Aug 04 16:44:01.455367 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.455251 2556 scope.go:117] "RemoveContainer" containerID="3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe" Aug 04 16:44:01.455571 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:44:01.455546 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe\": container with ID starting with 3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe not found: ID does not exist" containerID="3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe" Aug 04 16:44:01.455691 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.455578 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe"} err="failed to get container status \"3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe\": rpc error: code = NotFound desc = could not find container \"3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe\": container with ID starting with 3c3f061328690b983ccc971d80516fae0caf6cfb445381465a73a96689cfabbe not found: ID does not exist" Aug 04 16:44:01.455691 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.455596 2556 scope.go:117] "RemoveContainer" containerID="263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b" Aug 04 16:44:01.464174 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.464157 2556 scope.go:117] "RemoveContainer" containerID="263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b" Aug 04 16:44:01.464453 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:44:01.464430 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b\": container with ID starting with 263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b not found: ID does not exist" containerID="263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b" Aug 04 16:44:01.464524 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.464460 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b"} err="failed to get container status \"263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b\": rpc error: code = NotFound desc = could not find container \"263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b\": container with ID starting with 263eea85b5e23ddc93a13086334ea308eac5b916ab745d9e300ac38e6bcd2a6b not found: ID does not exist" Aug 04 16:44:01.472040 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.472010 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx"] Aug 04 16:44:01.476228 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.476201 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1f0f0c-769cc98d4d-jk6fx"] Aug 04 16:44:01.486705 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.486677 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k"] Aug 04 16:44:01.490311 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.490283 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1f0f0c-856757cbb4-djd9k"] Aug 04 16:44:01.493829 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.493808 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzjbr\" (UniqueName: \"kubernetes.io/projected/91937e79-47a7-4bf0-986e-01bca13362b8-kube-api-access-gzjbr\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:01.493931 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.493832 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91937e79-47a7-4bf0-986e-01bca13362b8-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:01.493931 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:01.493848 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmgt6\" (UniqueName: \"kubernetes.io/projected/6405d4d0-9add-444d-a3db-d9c17d9cd11e-kube-api-access-rmgt6\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:03.274028 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:03.273931 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6405d4d0-9add-444d-a3db-d9c17d9cd11e" path="/var/lib/kubelet/pods/6405d4d0-9add-444d-a3db-d9c17d9cd11e/volumes" Aug 04 16:44:03.274535 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:03.274514 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91937e79-47a7-4bf0-986e-01bca13362b8" path="/var/lib/kubelet/pods/91937e79-47a7-4bf0-986e-01bca13362b8/volumes" Aug 04 16:44:07.664593 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:07.664549 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr"] Aug 04 16:44:07.665094 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:07.664826 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" containerID="cri-o://c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d" gracePeriod=30 Aug 04 16:44:08.791121 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791086 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb"] Aug 04 16:44:08.791488 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791472 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" Aug 04 16:44:08.791488 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791483 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791499 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791504 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791512 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72732216-8b0c-4aee-8ad1-e97165365624" containerName="payload-pre-processing" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791518 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="72732216-8b0c-4aee-8ad1-e97165365624" containerName="payload-pre-processing" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791524 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6405d4d0-9add-444d-a3db-d9c17d9cd11e" containerName="payload-processing" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791530 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6405d4d0-9add-444d-a3db-d9c17d9cd11e" containerName="payload-processing" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791547 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91937e79-47a7-4bf0-986e-01bca13362b8" containerName="payload-pre-processing" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791552 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="91937e79-47a7-4bf0-986e-01bca13362b8" containerName="payload-pre-processing" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791561 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fc901b5-5983-42ec-be9b-bd4f9596534f" containerName="payload-processing" Aug 04 16:44:08.791572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791566 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fc901b5-5983-42ec-be9b-bd4f9596534f" containerName="payload-processing" Aug 04 16:44:08.791884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791622 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="91937e79-47a7-4bf0-986e-01bca13362b8" containerName="payload-pre-processing" Aug 04 16:44:08.791884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791630 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="72732216-8b0c-4aee-8ad1-e97165365624" containerName="payload-pre-processing" Aug 04 16:44:08.791884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791638 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" Aug 04 16:44:08.791884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791644 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bd8f7d0-1333-4633-b6c8-1815227a2f3c" containerName="istio-proxy" Aug 04 16:44:08.791884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791650 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6405d4d0-9add-444d-a3db-d9c17d9cd11e" containerName="payload-processing" Aug 04 16:44:08.791884 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.791656 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fc901b5-5983-42ec-be9b-bd4f9596534f" containerName="payload-processing" Aug 04 16:44:08.794754 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.794737 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.804787 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.804768 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-78e391-openshift-default-dockercfg-pwdl9\"" Aug 04 16:44:08.835201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.835170 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb"] Aug 04 16:44:08.864614 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.864470 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-token\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.864614 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.864529 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6cqg\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-kube-api-access-l6cqg\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.864614 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.864572 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-credential-socket\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.864614 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.864599 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-podinfo\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.865152 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.864644 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-envoy\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.865152 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.864776 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-data\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.865152 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.864842 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-certs\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.865152 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.864896 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-socket\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.865152 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.864923 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d16b18a8-c690-4c25-a8fc-126c99fcadca-istiod-ca-cert\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.965975 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.965935 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-token\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.965975 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.965973 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6cqg\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-kube-api-access-l6cqg\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.965995 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-credential-socket\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966012 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-podinfo\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966034 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-envoy\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966091 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-data\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966124 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-certs\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966177 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-socket\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966204 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d16b18a8-c690-4c25-a8fc-126c99fcadca-istiod-ca-cert\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966616 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966414 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-credential-socket\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966616 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966535 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-data\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966815 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966793 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-certs\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966902 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966803 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-socket\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.966902 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.966848 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d16b18a8-c690-4c25-a8fc-126c99fcadca-istiod-ca-cert\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.968982 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.968950 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-envoy\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.969095 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.969078 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-podinfo\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.976025 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.975999 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-token\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:08.976134 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:08.976048 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6cqg\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-kube-api-access-l6cqg\") pod \"e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:09.105461 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:09.105363 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:09.262153 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:09.262125 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb"] Aug 04 16:44:09.264313 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:44:09.264282 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd16b18a8_c690_4c25_a8fc_126c99fcadca.slice/crio-27a5456d209b7431fadd1812e866571c904aafcede0fb93d1aa465c487ace188 WatchSource:0}: Error finding container 27a5456d209b7431fadd1812e866571c904aafcede0fb93d1aa465c487ace188: Status 404 returned error can't find the container with id 27a5456d209b7431fadd1812e866571c904aafcede0fb93d1aa465c487ace188 Aug 04 16:44:09.266451 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:09.266421 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:09.266517 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:09.266488 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:09.266559 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:09.266517 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:09.491785 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:09.491749 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" event={"ID":"d16b18a8-c690-4c25-a8fc-126c99fcadca","Type":"ContainerStarted","Data":"ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237"} Aug 04 16:44:09.491970 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:09.491818 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" event={"ID":"d16b18a8-c690-4c25-a8fc-126c99fcadca","Type":"ContainerStarted","Data":"27a5456d209b7431fadd1812e866571c904aafcede0fb93d1aa465c487ace188"} Aug 04 16:44:09.518238 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:09.518181 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" podStartSLOduration=1.518162062 podStartE2EDuration="1.518162062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:44:09.516248437 +0000 UTC m=+2878.838671921" watchObservedRunningTime="2026-08-04 16:44:09.518162062 +0000 UTC m=+2878.840585541" Aug 04 16:44:10.106363 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:10.106325 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:10.111116 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:10.111090 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:10.497106 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:10.497079 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:10.498232 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:10.498212 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:12.929152 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:12.929086 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:44:13.006494 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006456 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-certs\") pod \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " Aug 04 16:44:13.006648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006516 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-socket\") pod \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " Aug 04 16:44:13.006648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006572 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-data\") pod \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " Aug 04 16:44:13.006648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006615 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istiod-ca-cert\") pod \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " Aug 04 16:44:13.006648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006646 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-envoy\") pod \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " Aug 04 16:44:13.006926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006688 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkv9m\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-kube-api-access-bkv9m\") pod \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " Aug 04 16:44:13.006926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006754 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-credential-socket\") pod \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " Aug 04 16:44:13.006926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006814 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-podinfo\") pod \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " Aug 04 16:44:13.006926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006822 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "119da46a-a7b6-4261-ae7a-dd9f1d1c4117" (UID: "119da46a-a7b6-4261-ae7a-dd9f1d1c4117"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:44:13.006926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006858 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-token\") pod \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\" (UID: \"119da46a-a7b6-4261-ae7a-dd9f1d1c4117\") " Aug 04 16:44:13.006926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.006875 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "119da46a-a7b6-4261-ae7a-dd9f1d1c4117" (UID: "119da46a-a7b6-4261-ae7a-dd9f1d1c4117"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:44:13.007206 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.007024 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-data" (OuterVolumeSpecName: "istio-data") pod "119da46a-a7b6-4261-ae7a-dd9f1d1c4117" (UID: "119da46a-a7b6-4261-ae7a-dd9f1d1c4117"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:44:13.007206 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.007175 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "119da46a-a7b6-4261-ae7a-dd9f1d1c4117" (UID: "119da46a-a7b6-4261-ae7a-dd9f1d1c4117"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:44:13.007319 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.007263 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:13.007319 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.007300 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:13.007319 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.007312 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:13.007462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.007325 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:13.007462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.007368 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "119da46a-a7b6-4261-ae7a-dd9f1d1c4117" (UID: "119da46a-a7b6-4261-ae7a-dd9f1d1c4117"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:44:13.009015 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.008986 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-kube-api-access-bkv9m" (OuterVolumeSpecName: "kube-api-access-bkv9m") pod "119da46a-a7b6-4261-ae7a-dd9f1d1c4117" (UID: "119da46a-a7b6-4261-ae7a-dd9f1d1c4117"). InnerVolumeSpecName "kube-api-access-bkv9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:44:13.009201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.009184 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-token" (OuterVolumeSpecName: "istio-token") pod "119da46a-a7b6-4261-ae7a-dd9f1d1c4117" (UID: "119da46a-a7b6-4261-ae7a-dd9f1d1c4117"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:44:13.009484 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.009464 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "119da46a-a7b6-4261-ae7a-dd9f1d1c4117" (UID: "119da46a-a7b6-4261-ae7a-dd9f1d1c4117"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:44:13.009524 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.009494 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "119da46a-a7b6-4261-ae7a-dd9f1d1c4117" (UID: "119da46a-a7b6-4261-ae7a-dd9f1d1c4117"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:44:13.108339 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.108300 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:13.108339 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.108332 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:13.108339 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.108342 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkv9m\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-kube-api-access-bkv9m\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:13.108339 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.108351 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:13.108612 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.108359 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/119da46a-a7b6-4261-ae7a-dd9f1d1c4117-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:44:13.515824 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.515791 2556 generic.go:356] "Generic (PLEG): container finished" podID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerID="c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d" exitCode=0 Aug 04 16:44:13.515994 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.515850 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" Aug 04 16:44:13.515994 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.515869 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" event={"ID":"119da46a-a7b6-4261-ae7a-dd9f1d1c4117","Type":"ContainerDied","Data":"c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d"} Aug 04 16:44:13.515994 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.515946 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr" event={"ID":"119da46a-a7b6-4261-ae7a-dd9f1d1c4117","Type":"ContainerDied","Data":"2b1489bc6dd9d346764c038d4db65c4729cc82822da9a0709a2d57cb4354f7e1"} Aug 04 16:44:13.515994 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.515968 2556 scope.go:117] "RemoveContainer" containerID="c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d" Aug 04 16:44:13.524876 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.524856 2556 scope.go:117] "RemoveContainer" containerID="b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6" Aug 04 16:44:13.532956 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.532938 2556 scope.go:117] "RemoveContainer" containerID="c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d" Aug 04 16:44:13.533208 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:44:13.533189 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d\": container with ID starting with c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d not found: ID does not exist" containerID="c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d" Aug 04 16:44:13.533250 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.533219 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d"} err="failed to get container status \"c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d\": rpc error: code = NotFound desc = could not find container \"c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d\": container with ID starting with c0c381f03c30a0016bf7b03ab58fa286e374e9369b93948595f22db726bcc45d not found: ID does not exist" Aug 04 16:44:13.533250 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.533239 2556 scope.go:117] "RemoveContainer" containerID="b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6" Aug 04 16:44:13.533461 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:44:13.533444 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6\": container with ID starting with b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6 not found: ID does not exist" containerID="b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6" Aug 04 16:44:13.533502 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.533468 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6"} err="failed to get container status \"b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6\": rpc error: code = NotFound desc = could not find container \"b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6\": container with ID starting with b114d492484328455aac8393586f34f369a6b407dadd742f4a7a996dab5d7cc6 not found: ID does not exist" Aug 04 16:44:13.538964 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.538939 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr"] Aug 04 16:44:13.543827 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.543806 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1f0f0c-openshift-default-645c7c4fc6-6m4vr"] Aug 04 16:44:13.585958 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.585923 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr"] Aug 04 16:44:13.586321 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.586308 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" Aug 04 16:44:13.586367 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.586324 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" Aug 04 16:44:13.586422 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.586411 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" Aug 04 16:44:13.590860 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.590844 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:13.593755 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.593717 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-78e391\"" Aug 04 16:44:13.593867 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.593740 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-78e391-dockercfg-vnhxv\"" Aug 04 16:44:13.600266 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.600237 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr"] Aug 04 16:44:13.653862 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.653799 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn"] Aug 04 16:44:13.654475 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.654446 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" Aug 04 16:44:13.654475 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.654473 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" Aug 04 16:44:13.654823 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.654589 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" containerName="istio-proxy" Aug 04 16:44:13.658187 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.658165 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:13.668059 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.668034 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn"] Aug 04 16:44:13.715113 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.715074 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvl28\" (UniqueName: \"kubernetes.io/projected/e033badd-bf96-44d8-bff5-f61a9f067dd0-kube-api-access-wvl28\") pod \"payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr\" (UID: \"e033badd-bf96-44d8-bff5-f61a9f067dd0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:13.715291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.715142 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e033badd-bf96-44d8-bff5-f61a9f067dd0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr\" (UID: \"e033badd-bf96-44d8-bff5-f61a9f067dd0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:13.715291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.715182 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e2da1d9-18d8-4009-8861-451a80e0a3b1-plugins-config-volume\") pod \"payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn\" (UID: \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\") " pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:13.715291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.715260 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld95j\" (UniqueName: \"kubernetes.io/projected/1e2da1d9-18d8-4009-8861-451a80e0a3b1-kube-api-access-ld95j\") pod \"payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn\" (UID: \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\") " pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:13.816351 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.816246 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvl28\" (UniqueName: \"kubernetes.io/projected/e033badd-bf96-44d8-bff5-f61a9f067dd0-kube-api-access-wvl28\") pod \"payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr\" (UID: \"e033badd-bf96-44d8-bff5-f61a9f067dd0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:13.816351 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.816325 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e033badd-bf96-44d8-bff5-f61a9f067dd0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr\" (UID: \"e033badd-bf96-44d8-bff5-f61a9f067dd0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:13.816690 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.816360 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e2da1d9-18d8-4009-8861-451a80e0a3b1-plugins-config-volume\") pod \"payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn\" (UID: \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\") " pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:13.816690 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.816416 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld95j\" (UniqueName: \"kubernetes.io/projected/1e2da1d9-18d8-4009-8861-451a80e0a3b1-kube-api-access-ld95j\") pod \"payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn\" (UID: \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\") " pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:13.817238 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.817210 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e033badd-bf96-44d8-bff5-f61a9f067dd0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr\" (UID: \"e033badd-bf96-44d8-bff5-f61a9f067dd0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:13.817410 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.817393 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e2da1d9-18d8-4009-8861-451a80e0a3b1-plugins-config-volume\") pod \"payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn\" (UID: \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\") " pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:13.825863 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.825835 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld95j\" (UniqueName: \"kubernetes.io/projected/1e2da1d9-18d8-4009-8861-451a80e0a3b1-kube-api-access-ld95j\") pod \"payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn\" (UID: \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\") " pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:13.825863 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.825854 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvl28\" (UniqueName: \"kubernetes.io/projected/e033badd-bf96-44d8-bff5-f61a9f067dd0-kube-api-access-wvl28\") pod \"payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr\" (UID: \"e033badd-bf96-44d8-bff5-f61a9f067dd0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:13.902964 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.902927 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:13.970201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:13.970167 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:14.057159 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.055031 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr"] Aug 04 16:44:14.130927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.130902 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn"] Aug 04 16:44:14.132665 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:44:14.132638 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e2da1d9_18d8_4009_8861_451a80e0a3b1.slice/crio-aa72d5c3b7f7a1e8bc64d2b9b8a7fe3cad81b10e2197b7fb1e4f6e57305b2c40 WatchSource:0}: Error finding container aa72d5c3b7f7a1e8bc64d2b9b8a7fe3cad81b10e2197b7fb1e4f6e57305b2c40: Status 404 returned error can't find the container with id aa72d5c3b7f7a1e8bc64d2b9b8a7fe3cad81b10e2197b7fb1e4f6e57305b2c40 Aug 04 16:44:14.525513 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.525479 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" event={"ID":"e033badd-bf96-44d8-bff5-f61a9f067dd0","Type":"ContainerStarted","Data":"89ddab28f96d6029229439fe66ac56ccc4fb7079209450ca2a64da1bcab0bc58"} Aug 04 16:44:14.525775 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.525531 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" event={"ID":"e033badd-bf96-44d8-bff5-f61a9f067dd0","Type":"ContainerStarted","Data":"ad431ca1c486835ca7c04259f7be68c38680e04f2df7166b070e21a1dcc04ac8"} Aug 04 16:44:14.525775 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.525590 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:14.527140 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.527116 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" event={"ID":"1e2da1d9-18d8-4009-8861-451a80e0a3b1","Type":"ContainerStarted","Data":"6cf6cba17a193bfd7a26c3e7f8d105f5203b296b620f21cae54dbd6f04b4d316"} Aug 04 16:44:14.527258 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.527162 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:14.527258 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.527188 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" event={"ID":"1e2da1d9-18d8-4009-8861-451a80e0a3b1","Type":"ContainerStarted","Data":"aa72d5c3b7f7a1e8bc64d2b9b8a7fe3cad81b10e2197b7fb1e4f6e57305b2c40"} Aug 04 16:44:14.543628 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.543581 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" podStartSLOduration=1.543567043 podStartE2EDuration="1.543567043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:44:14.542686553 +0000 UTC m=+2883.865110033" watchObservedRunningTime="2026-08-04 16:44:14.543567043 +0000 UTC m=+2883.865990523" Aug 04 16:44:14.563483 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:14.563424 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" podStartSLOduration=1.563407562 podStartE2EDuration="1.563407562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:44:14.561789679 +0000 UTC m=+2883.884213160" watchObservedRunningTime="2026-08-04 16:44:14.563407562 +0000 UTC m=+2883.885831041" Aug 04 16:44:15.274987 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:15.274947 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="119da46a-a7b6-4261-ae7a-dd9f1d1c4117" path="/var/lib/kubelet/pods/119da46a-a7b6-4261-ae7a-dd9f1d1c4117/volumes" Aug 04 16:44:24.583485 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:24.583456 2556 generic.go:356] "Generic (PLEG): container finished" podID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerID="ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237" exitCode=0 Aug 04 16:44:24.583845 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:24.583515 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" event={"ID":"d16b18a8-c690-4c25-a8fc-126c99fcadca","Type":"ContainerDied","Data":"ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237"} Aug 04 16:44:24.583909 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:24.583895 2556 scope.go:117] "RemoveContainer" containerID="ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237" Aug 04 16:44:24.584903 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:24.584873 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:24.584961 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:24.584946 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:24.584999 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:24.584980 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:25.497515 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:25.497462 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:25.535920 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:25.535891 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:44:25.536095 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:25.535958 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:44:25.608012 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:25.607974 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" event={"ID":"d16b18a8-c690-4c25-a8fc-126c99fcadca","Type":"ContainerStarted","Data":"69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943"} Aug 04 16:44:26.106298 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:26.106251 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:26.107782 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:26.107745 2556 patch_prober.go:28] interesting pod/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Aug 04 16:44:26.107927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:26.107810 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Aug 04 16:44:27.105829 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:27.105787 2556 patch_prober.go:28] interesting pod/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Aug 04 16:44:27.106238 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:27.105861 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Aug 04 16:44:28.106035 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:28.105996 2556 patch_prober.go:28] interesting pod/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Aug 04 16:44:28.106464 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:28.106052 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Aug 04 16:44:29.106022 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:29.105985 2556 patch_prober.go:28] interesting pod/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Aug 04 16:44:29.106236 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:29.106066 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Aug 04 16:44:30.106048 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:30.106003 2556 patch_prober.go:28] interesting pod/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Aug 04 16:44:30.106263 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:30.106068 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Aug 04 16:44:31.109824 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:31.109794 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:31.110273 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:31.110067 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:31.110842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:31.110824 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:44:37.387739 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.387694 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz"] Aug 04 16:44:37.391826 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.391803 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.394514 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.394488 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e13060-openshift-default-dockercfg-2px6j\"" Aug 04 16:44:37.410158 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.410121 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz"] Aug 04 16:44:37.460415 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.460371 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-data\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.460613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.460417 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02655b32-4a3f-4783-9f99-c725e23bf8f1-istiod-ca-cert\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.460613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.460477 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-credential-socket\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.460613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.460507 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-token\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.460613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.460533 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmjx2\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-kube-api-access-rmjx2\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.460613 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.460574 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-podinfo\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.460868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.460641 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-certs\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.460868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.460684 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-envoy\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.460868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.460741 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-socket\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.561404 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.561334 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-socket\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.561616 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.561450 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-data\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.561772 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.561484 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02655b32-4a3f-4783-9f99-c725e23bf8f1-istiod-ca-cert\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.561999 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.561823 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-socket\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562133 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.561878 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-data\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562232 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.561969 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-credential-socket\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562388 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.562322 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-credential-socket\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562388 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.562331 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-token\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562524 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.562394 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmjx2\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-kube-api-access-rmjx2\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562524 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.562438 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-podinfo\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562524 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.562493 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-certs\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562661 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.562525 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-envoy\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562661 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.562584 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02655b32-4a3f-4783-9f99-c725e23bf8f1-istiod-ca-cert\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.562868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.562848 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-certs\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.564821 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.564792 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-envoy\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.564980 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.564962 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-podinfo\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.572055 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.572030 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-token\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.572055 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.572047 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmjx2\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-kube-api-access-rmjx2\") pod \"e2e-api-b-e13060-openshift-default-7c78484864-d2hsz\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.703316 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.703226 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:37.842602 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.842578 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz"] Aug 04 16:44:37.844337 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:44:37.844310 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02655b32_4a3f_4783_9f99_c725e23bf8f1.slice/crio-0756f8d6a48a9f09f6b2b1433620b45074169ecd41ca6e63507154fc0ad7e201 WatchSource:0}: Error finding container 0756f8d6a48a9f09f6b2b1433620b45074169ecd41ca6e63507154fc0ad7e201: Status 404 returned error can't find the container with id 0756f8d6a48a9f09f6b2b1433620b45074169ecd41ca6e63507154fc0ad7e201 Aug 04 16:44:37.847113 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.847068 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:37.847215 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.847195 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:37.847261 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:37.847246 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:38.683059 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:38.683021 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" event={"ID":"02655b32-4a3f-4783-9f99-c725e23bf8f1","Type":"ContainerStarted","Data":"03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0"} Aug 04 16:44:38.683512 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:38.683100 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" event={"ID":"02655b32-4a3f-4783-9f99-c725e23bf8f1","Type":"ContainerStarted","Data":"0756f8d6a48a9f09f6b2b1433620b45074169ecd41ca6e63507154fc0ad7e201"} Aug 04 16:44:38.704353 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:38.704321 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:38.709350 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:38.709323 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:38.722106 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:38.722060 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" podStartSLOduration=1.722045267 podStartE2EDuration="1.722045267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:44:38.720386651 +0000 UTC m=+2908.042810145" watchObservedRunningTime="2026-08-04 16:44:38.722045267 +0000 UTC m=+2908.044468747" Aug 04 16:44:39.688498 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:39.688468 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:39.689343 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:39.689313 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:43.235038 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.235000 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd"] Aug 04 16:44:43.238918 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.238895 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:44:43.241440 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.241415 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-e13060\"" Aug 04 16:44:43.241581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.241534 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-e13060-dockercfg-jhmkw\"" Aug 04 16:44:43.251979 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.251940 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt"] Aug 04 16:44:43.256220 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.256194 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd"] Aug 04 16:44:43.256331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.256296 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:43.280445 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.280408 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt"] Aug 04 16:44:43.320362 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.320324 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f85446c-5dbd-413f-a124-bd06119bb07e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt\" (UID: \"3f85446c-5dbd-413f-a124-bd06119bb07e\") " pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:43.320536 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.320367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-642f4\" (UniqueName: \"kubernetes.io/projected/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-kube-api-access-642f4\") pod \"payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd\" (UID: \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:44:43.320536 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.320465 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w45rg\" (UniqueName: \"kubernetes.io/projected/3f85446c-5dbd-413f-a124-bd06119bb07e-kube-api-access-w45rg\") pod \"payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt\" (UID: \"3f85446c-5dbd-413f-a124-bd06119bb07e\") " pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:43.320628 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.320570 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd\" (UID: \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:44:43.422038 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.422001 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w45rg\" (UniqueName: \"kubernetes.io/projected/3f85446c-5dbd-413f-a124-bd06119bb07e-kube-api-access-w45rg\") pod \"payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt\" (UID: \"3f85446c-5dbd-413f-a124-bd06119bb07e\") " pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:43.422215 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.422081 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd\" (UID: \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:44:43.422215 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.422163 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f85446c-5dbd-413f-a124-bd06119bb07e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt\" (UID: \"3f85446c-5dbd-413f-a124-bd06119bb07e\") " pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:43.422215 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.422191 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-642f4\" (UniqueName: \"kubernetes.io/projected/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-kube-api-access-642f4\") pod \"payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd\" (UID: \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:44:43.422899 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.422872 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f85446c-5dbd-413f-a124-bd06119bb07e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt\" (UID: \"3f85446c-5dbd-413f-a124-bd06119bb07e\") " pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:43.423027 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.422920 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd\" (UID: \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:44:43.431577 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.431546 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-642f4\" (UniqueName: \"kubernetes.io/projected/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-kube-api-access-642f4\") pod \"payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd\" (UID: \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:44:43.431694 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.431604 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w45rg\" (UniqueName: \"kubernetes.io/projected/3f85446c-5dbd-413f-a124-bd06119bb07e-kube-api-access-w45rg\") pod \"payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt\" (UID: \"3f85446c-5dbd-413f-a124-bd06119bb07e\") " pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:43.556498 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.556460 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:44:43.566918 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.566886 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:43.708108 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.708074 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd"] Aug 04 16:44:43.709383 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:44:43.709332 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fd752b_dde5_4b6b_af91_795a1a6ddde3.slice/crio-ad88076ef77c773283ce8a1f122ca71f2a4ea0a2c1e15488a5501ce7bd100aae WatchSource:0}: Error finding container ad88076ef77c773283ce8a1f122ca71f2a4ea0a2c1e15488a5501ce7bd100aae: Status 404 returned error can't find the container with id ad88076ef77c773283ce8a1f122ca71f2a4ea0a2c1e15488a5501ce7bd100aae Aug 04 16:44:43.741826 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:43.741800 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt"] Aug 04 16:44:43.745009 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:44:43.744985 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f85446c_5dbd_413f_a124_bd06119bb07e.slice/crio-7715ee09531ea0e3c73819d29410752cbee6c5c47828c971bd4df18c8b7b917f WatchSource:0}: Error finding container 7715ee09531ea0e3c73819d29410752cbee6c5c47828c971bd4df18c8b7b917f: Status 404 returned error can't find the container with id 7715ee09531ea0e3c73819d29410752cbee6c5c47828c971bd4df18c8b7b917f Aug 04 16:44:44.719562 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:44.719495 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" event={"ID":"e7fd752b-dde5-4b6b-af91-795a1a6ddde3","Type":"ContainerStarted","Data":"010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd"} Aug 04 16:44:44.719562 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:44.719566 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:44:44.720112 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:44.719589 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" event={"ID":"e7fd752b-dde5-4b6b-af91-795a1a6ddde3","Type":"ContainerStarted","Data":"ad88076ef77c773283ce8a1f122ca71f2a4ea0a2c1e15488a5501ce7bd100aae"} Aug 04 16:44:44.721098 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:44.721072 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" event={"ID":"3f85446c-5dbd-413f-a124-bd06119bb07e","Type":"ContainerStarted","Data":"d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3"} Aug 04 16:44:44.721221 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:44.721112 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" event={"ID":"3f85446c-5dbd-413f-a124-bd06119bb07e","Type":"ContainerStarted","Data":"7715ee09531ea0e3c73819d29410752cbee6c5c47828c971bd4df18c8b7b917f"} Aug 04 16:44:44.721221 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:44.721132 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:44.736694 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:44.736642 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" podStartSLOduration=1.736625911 podStartE2EDuration="1.736625911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:44:44.734560746 +0000 UTC m=+2914.056984227" watchObservedRunningTime="2026-08-04 16:44:44.736625911 +0000 UTC m=+2914.059049417" Aug 04 16:44:44.750527 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:44.750486 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" podStartSLOduration=1.750469104 podStartE2EDuration="1.750469104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:44:44.750202362 +0000 UTC m=+2914.072625843" watchObservedRunningTime="2026-08-04 16:44:44.750469104 +0000 UTC m=+2914.072892588" Aug 04 16:44:55.728553 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:55.728505 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" podUID="e7fd752b-dde5-4b6b-af91-795a1a6ddde3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.74:9004: connect: connection refused" Aug 04 16:44:55.729013 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:55.728834 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:44:55.730858 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:55.730831 2556 patch_prober.go:28] interesting pod/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Aug 04 16:44:55.731016 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:55.730879 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Aug 04 16:44:57.800748 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:57.800698 2556 generic.go:356] "Generic (PLEG): container finished" podID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerID="03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0" exitCode=0 Aug 04 16:44:57.801198 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:57.800776 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" event={"ID":"02655b32-4a3f-4783-9f99-c725e23bf8f1","Type":"ContainerDied","Data":"03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0"} Aug 04 16:44:57.801260 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:57.801246 2556 scope.go:117] "RemoveContainer" containerID="03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0" Aug 04 16:44:57.802407 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:57.802378 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:57.802519 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:57.802463 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:57.802519 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:57.802510 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:44:58.808468 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:58.808435 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" event={"ID":"02655b32-4a3f-4783-9f99-c725e23bf8f1","Type":"ContainerStarted","Data":"7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145"} Aug 04 16:44:59.704474 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:59.704429 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:44:59.704710 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:59.704683 2556 patch_prober.go:28] interesting pod/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Aug 04 16:44:59.704845 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:44:59.704771 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Aug 04 16:45:00.704459 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:00.704422 2556 patch_prober.go:28] interesting pod/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Aug 04 16:45:00.704908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:00.704508 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Aug 04 16:45:01.704688 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:01.704646 2556 patch_prober.go:28] interesting pod/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Aug 04 16:45:01.705205 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:01.704754 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Aug 04 16:45:02.704113 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:02.704075 2556 patch_prober.go:28] interesting pod/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Aug 04 16:45:02.704306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:02.704135 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Aug 04 16:45:03.703680 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:03.703639 2556 patch_prober.go:28] interesting pod/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Aug 04 16:45:03.704086 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:03.703702 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Aug 04 16:45:04.708118 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:04.708084 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:45:04.708561 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:04.708361 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:45:04.709122 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:04.709102 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:45:05.728312 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:05.728279 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:45:11.690766 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:11.690717 2556 scope.go:117] "RemoveContainer" containerID="622d4daa692bb9e8e06a866473f8fddf831eb633aee53ce41e90bc42ce3bbe00" Aug 04 16:45:11.729121 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:11.726964 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn"] Aug 04 16:45:11.729121 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:11.727280 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" podUID="1e2da1d9-18d8-4009-8861-451a80e0a3b1" containerName="payload-processing" containerID="cri-o://6cf6cba17a193bfd7a26c3e7f8d105f5203b296b620f21cae54dbd6f04b4d316" gracePeriod=30 Aug 04 16:45:11.744013 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:11.743977 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr"] Aug 04 16:45:11.744259 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:11.744210 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" podUID="e033badd-bf96-44d8-bff5-f61a9f067dd0" containerName="payload-pre-processing" containerID="cri-o://89ddab28f96d6029229439fe66ac56ccc4fb7079209450ca2a64da1bcab0bc58" gracePeriod=30 Aug 04 16:45:11.895993 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:11.895690 2556 generic.go:356] "Generic (PLEG): container finished" podID="1e2da1d9-18d8-4009-8861-451a80e0a3b1" containerID="6cf6cba17a193bfd7a26c3e7f8d105f5203b296b620f21cae54dbd6f04b4d316" exitCode=0 Aug 04 16:45:11.895993 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:11.895950 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" event={"ID":"1e2da1d9-18d8-4009-8861-451a80e0a3b1","Type":"ContainerDied","Data":"6cf6cba17a193bfd7a26c3e7f8d105f5203b296b620f21cae54dbd6f04b4d316"} Aug 04 16:45:11.898560 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:11.898309 2556 generic.go:356] "Generic (PLEG): container finished" podID="e033badd-bf96-44d8-bff5-f61a9f067dd0" containerID="89ddab28f96d6029229439fe66ac56ccc4fb7079209450ca2a64da1bcab0bc58" exitCode=0 Aug 04 16:45:11.898560 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:11.898375 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" event={"ID":"e033badd-bf96-44d8-bff5-f61a9f067dd0","Type":"ContainerDied","Data":"89ddab28f96d6029229439fe66ac56ccc4fb7079209450ca2a64da1bcab0bc58"} Aug 04 16:45:12.048928 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.048898 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:45:12.051810 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.051789 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:45:12.144868 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.144838 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e2da1d9-18d8-4009-8861-451a80e0a3b1-plugins-config-volume\") pod \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\" (UID: \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\") " Aug 04 16:45:12.145063 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.144910 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld95j\" (UniqueName: \"kubernetes.io/projected/1e2da1d9-18d8-4009-8861-451a80e0a3b1-kube-api-access-ld95j\") pod \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\" (UID: \"1e2da1d9-18d8-4009-8861-451a80e0a3b1\") " Aug 04 16:45:12.145063 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.144965 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e033badd-bf96-44d8-bff5-f61a9f067dd0-plugins-config-volume\") pod \"e033badd-bf96-44d8-bff5-f61a9f067dd0\" (UID: \"e033badd-bf96-44d8-bff5-f61a9f067dd0\") " Aug 04 16:45:12.145063 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.145035 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvl28\" (UniqueName: \"kubernetes.io/projected/e033badd-bf96-44d8-bff5-f61a9f067dd0-kube-api-access-wvl28\") pod \"e033badd-bf96-44d8-bff5-f61a9f067dd0\" (UID: \"e033badd-bf96-44d8-bff5-f61a9f067dd0\") " Aug 04 16:45:12.145244 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.145218 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e2da1d9-18d8-4009-8861-451a80e0a3b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e2da1d9-18d8-4009-8861-451a80e0a3b1" (UID: "1e2da1d9-18d8-4009-8861-451a80e0a3b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:45:12.145384 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.145369 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e2da1d9-18d8-4009-8861-451a80e0a3b1-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:12.145453 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.145377 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e033badd-bf96-44d8-bff5-f61a9f067dd0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e033badd-bf96-44d8-bff5-f61a9f067dd0" (UID: "e033badd-bf96-44d8-bff5-f61a9f067dd0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:45:12.146984 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.146958 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e2da1d9-18d8-4009-8861-451a80e0a3b1-kube-api-access-ld95j" (OuterVolumeSpecName: "kube-api-access-ld95j") pod "1e2da1d9-18d8-4009-8861-451a80e0a3b1" (UID: "1e2da1d9-18d8-4009-8861-451a80e0a3b1"). InnerVolumeSpecName "kube-api-access-ld95j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:45:12.147101 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.147049 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e033badd-bf96-44d8-bff5-f61a9f067dd0-kube-api-access-wvl28" (OuterVolumeSpecName: "kube-api-access-wvl28") pod "e033badd-bf96-44d8-bff5-f61a9f067dd0" (UID: "e033badd-bf96-44d8-bff5-f61a9f067dd0"). InnerVolumeSpecName "kube-api-access-wvl28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:45:12.246103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.246068 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld95j\" (UniqueName: \"kubernetes.io/projected/1e2da1d9-18d8-4009-8861-451a80e0a3b1-kube-api-access-ld95j\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:12.246103 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.246100 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e033badd-bf96-44d8-bff5-f61a9f067dd0-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:12.246299 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.246115 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvl28\" (UniqueName: \"kubernetes.io/projected/e033badd-bf96-44d8-bff5-f61a9f067dd0-kube-api-access-wvl28\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:12.905418 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.905385 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" Aug 04 16:45:12.905860 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.905430 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn" event={"ID":"1e2da1d9-18d8-4009-8861-451a80e0a3b1","Type":"ContainerDied","Data":"aa72d5c3b7f7a1e8bc64d2b9b8a7fe3cad81b10e2197b7fb1e4f6e57305b2c40"} Aug 04 16:45:12.905860 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.905499 2556 scope.go:117] "RemoveContainer" containerID="6cf6cba17a193bfd7a26c3e7f8d105f5203b296b620f21cae54dbd6f04b4d316" Aug 04 16:45:12.906926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.906904 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" event={"ID":"e033badd-bf96-44d8-bff5-f61a9f067dd0","Type":"ContainerDied","Data":"ad431ca1c486835ca7c04259f7be68c38680e04f2df7166b070e21a1dcc04ac8"} Aug 04 16:45:12.907025 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.906916 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr" Aug 04 16:45:12.915283 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.915105 2556 scope.go:117] "RemoveContainer" containerID="89ddab28f96d6029229439fe66ac56ccc4fb7079209450ca2a64da1bcab0bc58" Aug 04 16:45:12.930457 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.930432 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn"] Aug 04 16:45:12.934396 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.934375 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78e391-6895bbdb58-7xnzn"] Aug 04 16:45:12.944433 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.944407 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr"] Aug 04 16:45:12.951334 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:12.951303 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-78e391-658c7d5c9b-hgtzr"] Aug 04 16:45:13.274051 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:13.274019 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e2da1d9-18d8-4009-8861-451a80e0a3b1" path="/var/lib/kubelet/pods/1e2da1d9-18d8-4009-8861-451a80e0a3b1/volumes" Aug 04 16:45:13.274396 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:13.274383 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e033badd-bf96-44d8-bff5-f61a9f067dd0" path="/var/lib/kubelet/pods/e033badd-bf96-44d8-bff5-f61a9f067dd0/volumes" Aug 04 16:45:18.655838 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:18.655804 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb"] Aug 04 16:45:18.656331 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:18.656019 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" containerID="cri-o://69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943" gracePeriod=30 Aug 04 16:45:23.919906 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.919837 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:45:23.977306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.977275 2556 generic.go:356] "Generic (PLEG): container finished" podID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerID="69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943" exitCode=0 Aug 04 16:45:23.977306 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.977315 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" event={"ID":"d16b18a8-c690-4c25-a8fc-126c99fcadca","Type":"ContainerDied","Data":"69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943"} Aug 04 16:45:23.977581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.977334 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" Aug 04 16:45:23.977581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.977368 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb" event={"ID":"d16b18a8-c690-4c25-a8fc-126c99fcadca","Type":"ContainerDied","Data":"27a5456d209b7431fadd1812e866571c904aafcede0fb93d1aa465c487ace188"} Aug 04 16:45:23.977581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.977397 2556 scope.go:117] "RemoveContainer" containerID="69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943" Aug 04 16:45:23.988381 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.988355 2556 scope.go:117] "RemoveContainer" containerID="ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237" Aug 04 16:45:23.996795 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.996766 2556 scope.go:117] "RemoveContainer" containerID="69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943" Aug 04 16:45:23.997066 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:45:23.997044 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943\": container with ID starting with 69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943 not found: ID does not exist" containerID="69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943" Aug 04 16:45:23.997156 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.997080 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943"} err="failed to get container status \"69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943\": rpc error: code = NotFound desc = could not find container \"69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943\": container with ID starting with 69e24bd419d114446ff749a8851ab30d1032cac2ee2a3d07441840512457f943 not found: ID does not exist" Aug 04 16:45:23.997156 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.997107 2556 scope.go:117] "RemoveContainer" containerID="ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237" Aug 04 16:45:23.997386 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:45:23.997370 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237\": container with ID starting with ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237 not found: ID does not exist" containerID="ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237" Aug 04 16:45:23.997440 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:23.997397 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237"} err="failed to get container status \"ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237\": rpc error: code = NotFound desc = could not find container \"ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237\": container with ID starting with ecd9c09c2b24912bbde5a004c1ee76ab99d90a07c987d84e5206d589a4bc5237 not found: ID does not exist" Aug 04 16:45:24.067766 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.067707 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6cqg\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-kube-api-access-l6cqg\") pod \"d16b18a8-c690-4c25-a8fc-126c99fcadca\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " Aug 04 16:45:24.067766 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.067780 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-token\") pod \"d16b18a8-c690-4c25-a8fc-126c99fcadca\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " Aug 04 16:45:24.068070 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.067839 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d16b18a8-c690-4c25-a8fc-126c99fcadca-istiod-ca-cert\") pod \"d16b18a8-c690-4c25-a8fc-126c99fcadca\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " Aug 04 16:45:24.068070 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.067872 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-socket\") pod \"d16b18a8-c690-4c25-a8fc-126c99fcadca\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " Aug 04 16:45:24.068070 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.067908 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-certs\") pod \"d16b18a8-c690-4c25-a8fc-126c99fcadca\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " Aug 04 16:45:24.068070 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.067949 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-credential-socket\") pod \"d16b18a8-c690-4c25-a8fc-126c99fcadca\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " Aug 04 16:45:24.068070 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.067985 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-envoy\") pod \"d16b18a8-c690-4c25-a8fc-126c99fcadca\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " Aug 04 16:45:24.068070 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068059 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-podinfo\") pod \"d16b18a8-c690-4c25-a8fc-126c99fcadca\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " Aug 04 16:45:24.068366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068087 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-data\") pod \"d16b18a8-c690-4c25-a8fc-126c99fcadca\" (UID: \"d16b18a8-c690-4c25-a8fc-126c99fcadca\") " Aug 04 16:45:24.068366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068234 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d16b18a8-c690-4c25-a8fc-126c99fcadca" (UID: "d16b18a8-c690-4c25-a8fc-126c99fcadca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:24.068366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068242 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d16b18a8-c690-4c25-a8fc-126c99fcadca" (UID: "d16b18a8-c690-4c25-a8fc-126c99fcadca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:24.068366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068263 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d16b18a8-c690-4c25-a8fc-126c99fcadca" (UID: "d16b18a8-c690-4c25-a8fc-126c99fcadca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:24.068520 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068462 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-data" (OuterVolumeSpecName: "istio-data") pod "d16b18a8-c690-4c25-a8fc-126c99fcadca" (UID: "d16b18a8-c690-4c25-a8fc-126c99fcadca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:24.068576 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068515 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:24.068576 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068539 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:24.068576 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068554 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:24.068761 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.068720 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d16b18a8-c690-4c25-a8fc-126c99fcadca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d16b18a8-c690-4c25-a8fc-126c99fcadca" (UID: "d16b18a8-c690-4c25-a8fc-126c99fcadca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:45:24.070274 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.070244 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-kube-api-access-l6cqg" (OuterVolumeSpecName: "kube-api-access-l6cqg") pod "d16b18a8-c690-4c25-a8fc-126c99fcadca" (UID: "d16b18a8-c690-4c25-a8fc-126c99fcadca"). InnerVolumeSpecName "kube-api-access-l6cqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:45:24.070274 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.070255 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-token" (OuterVolumeSpecName: "istio-token") pod "d16b18a8-c690-4c25-a8fc-126c99fcadca" (UID: "d16b18a8-c690-4c25-a8fc-126c99fcadca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:45:24.070421 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.070285 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d16b18a8-c690-4c25-a8fc-126c99fcadca" (UID: "d16b18a8-c690-4c25-a8fc-126c99fcadca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:45:24.070498 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.070479 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d16b18a8-c690-4c25-a8fc-126c99fcadca" (UID: "d16b18a8-c690-4c25-a8fc-126c99fcadca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:24.170098 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.170012 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:24.170098 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.170039 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:24.170098 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.170048 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:24.170098 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.170059 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6cqg\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-kube-api-access-l6cqg\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:24.170098 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.170068 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d16b18a8-c690-4c25-a8fc-126c99fcadca-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:24.170098 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.170076 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d16b18a8-c690-4c25-a8fc-126c99fcadca-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:24.302294 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.302264 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb"] Aug 04 16:45:24.308455 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:24.308429 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-78e391-openshift-default-59f87fbc98-sm4pb"] Aug 04 16:45:25.274715 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:25.274683 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" path="/var/lib/kubelet/pods/d16b18a8-c690-4c25-a8fc-126c99fcadca/volumes" Aug 04 16:45:30.428917 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.428879 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt"] Aug 04 16:45:30.429356 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.429161 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" podUID="3f85446c-5dbd-413f-a124-bd06119bb07e" containerName="payload-processing" containerID="cri-o://d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3" gracePeriod=30 Aug 04 16:45:30.434434 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.433851 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd"] Aug 04 16:45:30.434434 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.434085 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" podUID="e7fd752b-dde5-4b6b-af91-795a1a6ddde3" containerName="payload-pre-processing" containerID="cri-o://010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd" gracePeriod=30 Aug 04 16:45:30.720850 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.720825 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:45:30.735200 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.735174 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:45:30.836485 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.836450 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w45rg\" (UniqueName: \"kubernetes.io/projected/3f85446c-5dbd-413f-a124-bd06119bb07e-kube-api-access-w45rg\") pod \"3f85446c-5dbd-413f-a124-bd06119bb07e\" (UID: \"3f85446c-5dbd-413f-a124-bd06119bb07e\") " Aug 04 16:45:30.836672 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.836508 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-642f4\" (UniqueName: \"kubernetes.io/projected/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-kube-api-access-642f4\") pod \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\" (UID: \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\") " Aug 04 16:45:30.836672 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.836574 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-plugins-config-volume\") pod \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\" (UID: \"e7fd752b-dde5-4b6b-af91-795a1a6ddde3\") " Aug 04 16:45:30.836672 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.836612 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f85446c-5dbd-413f-a124-bd06119bb07e-plugins-config-volume\") pod \"3f85446c-5dbd-413f-a124-bd06119bb07e\" (UID: \"3f85446c-5dbd-413f-a124-bd06119bb07e\") " Aug 04 16:45:30.837033 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.837007 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e7fd752b-dde5-4b6b-af91-795a1a6ddde3" (UID: "e7fd752b-dde5-4b6b-af91-795a1a6ddde3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:45:30.837096 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.837004 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f85446c-5dbd-413f-a124-bd06119bb07e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f85446c-5dbd-413f-a124-bd06119bb07e" (UID: "3f85446c-5dbd-413f-a124-bd06119bb07e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:45:30.838590 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.838563 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f85446c-5dbd-413f-a124-bd06119bb07e-kube-api-access-w45rg" (OuterVolumeSpecName: "kube-api-access-w45rg") pod "3f85446c-5dbd-413f-a124-bd06119bb07e" (UID: "3f85446c-5dbd-413f-a124-bd06119bb07e"). InnerVolumeSpecName "kube-api-access-w45rg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:45:30.838652 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.838617 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-kube-api-access-642f4" (OuterVolumeSpecName: "kube-api-access-642f4") pod "e7fd752b-dde5-4b6b-af91-795a1a6ddde3" (UID: "e7fd752b-dde5-4b6b-af91-795a1a6ddde3"). InnerVolumeSpecName "kube-api-access-642f4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:45:30.938390 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.938345 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w45rg\" (UniqueName: \"kubernetes.io/projected/3f85446c-5dbd-413f-a124-bd06119bb07e-kube-api-access-w45rg\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:30.938390 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.938382 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-642f4\" (UniqueName: \"kubernetes.io/projected/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-kube-api-access-642f4\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:30.938390 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.938395 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7fd752b-dde5-4b6b-af91-795a1a6ddde3-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:30.938645 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:30.938406 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f85446c-5dbd-413f-a124-bd06119bb07e-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:31.015432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.015392 2556 generic.go:356] "Generic (PLEG): container finished" podID="3f85446c-5dbd-413f-a124-bd06119bb07e" containerID="d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3" exitCode=0 Aug 04 16:45:31.015611 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.015444 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" Aug 04 16:45:31.015611 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.015477 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" event={"ID":"3f85446c-5dbd-413f-a124-bd06119bb07e","Type":"ContainerDied","Data":"d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3"} Aug 04 16:45:31.015611 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.015535 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt" event={"ID":"3f85446c-5dbd-413f-a124-bd06119bb07e","Type":"ContainerDied","Data":"7715ee09531ea0e3c73819d29410752cbee6c5c47828c971bd4df18c8b7b917f"} Aug 04 16:45:31.015611 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.015559 2556 scope.go:117] "RemoveContainer" containerID="d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3" Aug 04 16:45:31.017003 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.016979 2556 generic.go:356] "Generic (PLEG): container finished" podID="e7fd752b-dde5-4b6b-af91-795a1a6ddde3" containerID="010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd" exitCode=0 Aug 04 16:45:31.017129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.017008 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" event={"ID":"e7fd752b-dde5-4b6b-af91-795a1a6ddde3","Type":"ContainerDied","Data":"010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd"} Aug 04 16:45:31.017129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.017035 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" Aug 04 16:45:31.017129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.017062 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd" event={"ID":"e7fd752b-dde5-4b6b-af91-795a1a6ddde3","Type":"ContainerDied","Data":"ad88076ef77c773283ce8a1f122ca71f2a4ea0a2c1e15488a5501ce7bd100aae"} Aug 04 16:45:31.024674 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.024644 2556 scope.go:117] "RemoveContainer" containerID="d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3" Aug 04 16:45:31.024974 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:45:31.024949 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3\": container with ID starting with d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3 not found: ID does not exist" containerID="d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3" Aug 04 16:45:31.025072 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.024985 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3"} err="failed to get container status \"d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3\": rpc error: code = NotFound desc = could not find container \"d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3\": container with ID starting with d1c088b1ceb5b4a2a018e9bfaf3f143e045ec9b07ea12011fc460c4a3e1ab7f3 not found: ID does not exist" Aug 04 16:45:31.025072 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.025012 2556 scope.go:117] "RemoveContainer" containerID="010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd" Aug 04 16:45:31.033527 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.033509 2556 scope.go:117] "RemoveContainer" containerID="010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd" Aug 04 16:45:31.033797 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:45:31.033777 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd\": container with ID starting with 010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd not found: ID does not exist" containerID="010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd" Aug 04 16:45:31.033853 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.033805 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd"} err="failed to get container status \"010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd\": rpc error: code = NotFound desc = could not find container \"010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd\": container with ID starting with 010a8b897630a14cc438a2aba0e1e3635e90967c2de26a51844fe8f7b670c3dd not found: ID does not exist" Aug 04 16:45:31.042388 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.042363 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd"] Aug 04 16:45:31.047997 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.047975 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e13060-5dc9f4fbf6-d9zkd"] Aug 04 16:45:31.061380 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.061357 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt"] Aug 04 16:45:31.066736 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.066702 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e13060-69fd849b48-pmjmt"] Aug 04 16:45:31.282030 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.281949 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f85446c-5dbd-413f-a124-bd06119bb07e" path="/var/lib/kubelet/pods/3f85446c-5dbd-413f-a124-bd06119bb07e/volumes" Aug 04 16:45:31.282309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:31.282296 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7fd752b-dde5-4b6b-af91-795a1a6ddde3" path="/var/lib/kubelet/pods/e7fd752b-dde5-4b6b-af91-795a1a6ddde3/volumes" Aug 04 16:45:36.992653 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:36.992569 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz"] Aug 04 16:45:36.993219 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:36.992842 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" containerID="cri-o://7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145" gracePeriod=30 Aug 04 16:45:37.916947 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.916908 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5"] Aug 04 16:45:37.917336 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917322 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" Aug 04 16:45:37.917399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917339 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" Aug 04 16:45:37.917399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917352 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" Aug 04 16:45:37.917399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917358 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" Aug 04 16:45:37.917399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917366 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7fd752b-dde5-4b6b-af91-795a1a6ddde3" containerName="payload-pre-processing" Aug 04 16:45:37.917399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917372 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7fd752b-dde5-4b6b-af91-795a1a6ddde3" containerName="payload-pre-processing" Aug 04 16:45:37.917399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917378 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f85446c-5dbd-413f-a124-bd06119bb07e" containerName="payload-processing" Aug 04 16:45:37.917399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917383 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f85446c-5dbd-413f-a124-bd06119bb07e" containerName="payload-processing" Aug 04 16:45:37.917399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917394 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e033badd-bf96-44d8-bff5-f61a9f067dd0" containerName="payload-pre-processing" Aug 04 16:45:37.917399 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917399 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e033badd-bf96-44d8-bff5-f61a9f067dd0" containerName="payload-pre-processing" Aug 04 16:45:37.917852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917424 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e2da1d9-18d8-4009-8861-451a80e0a3b1" containerName="payload-processing" Aug 04 16:45:37.917852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917430 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e2da1d9-18d8-4009-8861-451a80e0a3b1" containerName="payload-processing" Aug 04 16:45:37.917852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917483 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" Aug 04 16:45:37.917852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917494 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e033badd-bf96-44d8-bff5-f61a9f067dd0" containerName="payload-pre-processing" Aug 04 16:45:37.917852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917503 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7fd752b-dde5-4b6b-af91-795a1a6ddde3" containerName="payload-pre-processing" Aug 04 16:45:37.917852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917511 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d16b18a8-c690-4c25-a8fc-126c99fcadca" containerName="istio-proxy" Aug 04 16:45:37.917852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917518 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f85446c-5dbd-413f-a124-bd06119bb07e" containerName="payload-processing" Aug 04 16:45:37.917852 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.917525 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e2da1d9-18d8-4009-8861-451a80e0a3b1" containerName="payload-processing" Aug 04 16:45:37.921678 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.921657 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:37.924418 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.924394 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-90d707-openshift-default-dockercfg-qfrf5\"" Aug 04 16:45:37.933423 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:37.933392 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5"] Aug 04 16:45:38.109208 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.109157 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-socket\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.109657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.109248 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr5jw\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-kube-api-access-fr5jw\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.109657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.109274 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-envoy\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.109657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.109309 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-certs\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.109657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.109395 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-podinfo\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.109657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.109427 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-token\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.109657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.109445 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acfbe88c-858a-47f4-a927-c69fe0fedb55-istiod-ca-cert\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.109657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.109529 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-data\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.109657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.109564 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-credential-socket\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211064 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.210961 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-envoy\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211064 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211008 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-certs\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211064 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211047 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-podinfo\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211362 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211073 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-token\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211362 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211096 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acfbe88c-858a-47f4-a927-c69fe0fedb55-istiod-ca-cert\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211362 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211153 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-data\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211362 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211198 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-credential-socket\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211362 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211255 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-socket\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211362 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211330 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr5jw\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-kube-api-access-fr5jw\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211695 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211448 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-certs\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211695 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211534 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-data\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211849 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211712 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-socket\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.211849 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211720 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-credential-socket\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.212000 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.211977 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acfbe88c-858a-47f4-a927-c69fe0fedb55-istiod-ca-cert\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.213444 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.213418 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-envoy\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.213656 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.213640 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-podinfo\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.219893 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.219858 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr5jw\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-kube-api-access-fr5jw\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.219893 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.219886 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-token\") pod \"e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.234898 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.234862 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:38.390411 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.390387 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5"] Aug 04 16:45:38.393444 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:45:38.393414 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacfbe88c_858a_47f4_a927_c69fe0fedb55.slice/crio-67930c8315c082a8e999b8f73095432519c0ea45d665074af5c6e7711bde79ee WatchSource:0}: Error finding container 67930c8315c082a8e999b8f73095432519c0ea45d665074af5c6e7711bde79ee: Status 404 returned error can't find the container with id 67930c8315c082a8e999b8f73095432519c0ea45d665074af5c6e7711bde79ee Aug 04 16:45:38.395521 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.395488 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:45:38.395624 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.395554 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:45:38.395624 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:38.395586 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:45:39.063649 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:39.063605 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" event={"ID":"acfbe88c-858a-47f4-a927-c69fe0fedb55","Type":"ContainerStarted","Data":"52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461"} Aug 04 16:45:39.063887 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:39.063671 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" event={"ID":"acfbe88c-858a-47f4-a927-c69fe0fedb55","Type":"ContainerStarted","Data":"67930c8315c082a8e999b8f73095432519c0ea45d665074af5c6e7711bde79ee"} Aug 04 16:45:39.087497 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:39.087441 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" podStartSLOduration=2.087425665 podStartE2EDuration="2.087425665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:45:39.086246735 +0000 UTC m=+2968.408670241" watchObservedRunningTime="2026-08-04 16:45:39.087425665 +0000 UTC m=+2968.409849145" Aug 04 16:45:39.235983 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:39.235947 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:39.241076 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:39.241047 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:40.068577 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:40.068539 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:40.069557 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:40.069540 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:42.257321 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.257252 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:45:42.352818 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.352782 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-credential-socket\") pod \"02655b32-4a3f-4783-9f99-c725e23bf8f1\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " Aug 04 16:45:42.353002 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.352866 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-certs\") pod \"02655b32-4a3f-4783-9f99-c725e23bf8f1\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " Aug 04 16:45:42.353002 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.352895 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-token\") pod \"02655b32-4a3f-4783-9f99-c725e23bf8f1\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " Aug 04 16:45:42.353002 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.352924 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-envoy\") pod \"02655b32-4a3f-4783-9f99-c725e23bf8f1\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " Aug 04 16:45:42.353002 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.352961 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-socket\") pod \"02655b32-4a3f-4783-9f99-c725e23bf8f1\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " Aug 04 16:45:42.353002 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353002 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02655b32-4a3f-4783-9f99-c725e23bf8f1-istiod-ca-cert\") pod \"02655b32-4a3f-4783-9f99-c725e23bf8f1\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " Aug 04 16:45:42.353277 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353062 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-data\") pod \"02655b32-4a3f-4783-9f99-c725e23bf8f1\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " Aug 04 16:45:42.353277 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353056 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "02655b32-4a3f-4783-9f99-c725e23bf8f1" (UID: "02655b32-4a3f-4783-9f99-c725e23bf8f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:42.353277 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353094 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmjx2\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-kube-api-access-rmjx2\") pod \"02655b32-4a3f-4783-9f99-c725e23bf8f1\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " Aug 04 16:45:42.353277 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353129 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-podinfo\") pod \"02655b32-4a3f-4783-9f99-c725e23bf8f1\" (UID: \"02655b32-4a3f-4783-9f99-c725e23bf8f1\") " Aug 04 16:45:42.353501 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353286 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "02655b32-4a3f-4783-9f99-c725e23bf8f1" (UID: "02655b32-4a3f-4783-9f99-c725e23bf8f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:42.353501 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353331 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "02655b32-4a3f-4783-9f99-c725e23bf8f1" (UID: "02655b32-4a3f-4783-9f99-c725e23bf8f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:42.353501 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353396 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02655b32-4a3f-4783-9f99-c725e23bf8f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "02655b32-4a3f-4783-9f99-c725e23bf8f1" (UID: "02655b32-4a3f-4783-9f99-c725e23bf8f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:45:42.353501 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353465 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "02655b32-4a3f-4783-9f99-c725e23bf8f1" (UID: "02655b32-4a3f-4783-9f99-c725e23bf8f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:42.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353524 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:42.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353542 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:42.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353556 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:42.353788 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.353569 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02655b32-4a3f-4783-9f99-c725e23bf8f1-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:42.355219 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.355187 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "02655b32-4a3f-4783-9f99-c725e23bf8f1" (UID: "02655b32-4a3f-4783-9f99-c725e23bf8f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:45:42.355340 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.355268 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-kube-api-access-rmjx2" (OuterVolumeSpecName: "kube-api-access-rmjx2") pod "02655b32-4a3f-4783-9f99-c725e23bf8f1" (UID: "02655b32-4a3f-4783-9f99-c725e23bf8f1"). InnerVolumeSpecName "kube-api-access-rmjx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:45:42.355340 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.355300 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "02655b32-4a3f-4783-9f99-c725e23bf8f1" (UID: "02655b32-4a3f-4783-9f99-c725e23bf8f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:45:42.355340 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.355317 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "02655b32-4a3f-4783-9f99-c725e23bf8f1" (UID: "02655b32-4a3f-4783-9f99-c725e23bf8f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:45:42.454960 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.454924 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:42.454960 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.454952 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:42.454960 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.454964 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmjx2\" (UniqueName: \"kubernetes.io/projected/02655b32-4a3f-4783-9f99-c725e23bf8f1-kube-api-access-rmjx2\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:42.455201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.454977 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02655b32-4a3f-4783-9f99-c725e23bf8f1-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:42.455201 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.454985 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02655b32-4a3f-4783-9f99-c725e23bf8f1-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:45:42.755880 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.755846 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8"] Aug 04 16:45:42.756469 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.756447 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" Aug 04 16:45:42.756469 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.756471 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" Aug 04 16:45:42.756656 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.756599 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" Aug 04 16:45:42.756656 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.756611 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" Aug 04 16:45:42.760067 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.760046 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:42.762418 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.762396 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-90d707\"" Aug 04 16:45:42.762541 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.762397 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-90d707-dockercfg-wlbx5\"" Aug 04 16:45:42.769357 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.769331 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8"] Aug 04 16:45:42.771532 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.771505 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28"] Aug 04 16:45:42.772083 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.772060 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" Aug 04 16:45:42.772179 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.772136 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerName="istio-proxy" Aug 04 16:45:42.775484 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.775464 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:42.783955 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.783932 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28"] Aug 04 16:45:42.858573 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.858534 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d910e5c0-7f63-4a32-9956-ab6019773273-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8\" (UID: \"d910e5c0-7f63-4a32-9956-ab6019773273\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:42.858781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.858637 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8mk5\" (UniqueName: \"kubernetes.io/projected/37e2aa30-834e-4578-a6b3-239e3456f69f-kube-api-access-f8mk5\") pod \"payload-processing-e2e-shared-a-90d707-56b986d878-rnx28\" (UID: \"37e2aa30-834e-4578-a6b3-239e3456f69f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:42.858781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.858672 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e2aa30-834e-4578-a6b3-239e3456f69f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-90d707-56b986d878-rnx28\" (UID: \"37e2aa30-834e-4578-a6b3-239e3456f69f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:42.858781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.858763 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8hdm\" (UniqueName: \"kubernetes.io/projected/d910e5c0-7f63-4a32-9956-ab6019773273-kube-api-access-c8hdm\") pod \"payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8\" (UID: \"d910e5c0-7f63-4a32-9956-ab6019773273\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:42.960010 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.959956 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8mk5\" (UniqueName: \"kubernetes.io/projected/37e2aa30-834e-4578-a6b3-239e3456f69f-kube-api-access-f8mk5\") pod \"payload-processing-e2e-shared-a-90d707-56b986d878-rnx28\" (UID: \"37e2aa30-834e-4578-a6b3-239e3456f69f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:42.960010 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.960022 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e2aa30-834e-4578-a6b3-239e3456f69f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-90d707-56b986d878-rnx28\" (UID: \"37e2aa30-834e-4578-a6b3-239e3456f69f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:42.960284 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.960080 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8hdm\" (UniqueName: \"kubernetes.io/projected/d910e5c0-7f63-4a32-9956-ab6019773273-kube-api-access-c8hdm\") pod \"payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8\" (UID: \"d910e5c0-7f63-4a32-9956-ab6019773273\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:42.960284 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.960117 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d910e5c0-7f63-4a32-9956-ab6019773273-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8\" (UID: \"d910e5c0-7f63-4a32-9956-ab6019773273\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:42.960908 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.960880 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e2aa30-834e-4578-a6b3-239e3456f69f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-90d707-56b986d878-rnx28\" (UID: \"37e2aa30-834e-4578-a6b3-239e3456f69f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:42.961007 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.960884 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d910e5c0-7f63-4a32-9956-ab6019773273-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8\" (UID: \"d910e5c0-7f63-4a32-9956-ab6019773273\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:42.969436 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.969406 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8hdm\" (UniqueName: \"kubernetes.io/projected/d910e5c0-7f63-4a32-9956-ab6019773273-kube-api-access-c8hdm\") pod \"payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8\" (UID: \"d910e5c0-7f63-4a32-9956-ab6019773273\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:42.969436 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:42.969423 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8mk5\" (UniqueName: \"kubernetes.io/projected/37e2aa30-834e-4578-a6b3-239e3456f69f-kube-api-access-f8mk5\") pod \"payload-processing-e2e-shared-a-90d707-56b986d878-rnx28\" (UID: \"37e2aa30-834e-4578-a6b3-239e3456f69f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:43.072745 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.072631 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:43.087612 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.087581 2556 generic.go:356] "Generic (PLEG): container finished" podID="02655b32-4a3f-4783-9f99-c725e23bf8f1" containerID="7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145" exitCode=0 Aug 04 16:45:43.087757 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.087641 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" Aug 04 16:45:43.087757 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.087669 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" event={"ID":"02655b32-4a3f-4783-9f99-c725e23bf8f1","Type":"ContainerDied","Data":"7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145"} Aug 04 16:45:43.087846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.087756 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz" event={"ID":"02655b32-4a3f-4783-9f99-c725e23bf8f1","Type":"ContainerDied","Data":"0756f8d6a48a9f09f6b2b1433620b45074169ecd41ca6e63507154fc0ad7e201"} Aug 04 16:45:43.087846 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.087794 2556 scope.go:117] "RemoveContainer" containerID="7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145" Aug 04 16:45:43.088280 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.088088 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:43.124676 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.124605 2556 scope.go:117] "RemoveContainer" containerID="03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0" Aug 04 16:45:43.138069 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.136361 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz"] Aug 04 16:45:43.138529 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.138454 2556 scope.go:117] "RemoveContainer" containerID="7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145" Aug 04 16:45:43.139256 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:45:43.139227 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145\": container with ID starting with 7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145 not found: ID does not exist" containerID="7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145" Aug 04 16:45:43.139437 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.139409 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145"} err="failed to get container status \"7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145\": rpc error: code = NotFound desc = could not find container \"7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145\": container with ID starting with 7a7c920fd1319e2232de2d61cedd7378f4a45ac0633b0d7d1898f17b7d41c145 not found: ID does not exist" Aug 04 16:45:43.139544 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.139532 2556 scope.go:117] "RemoveContainer" containerID="03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0" Aug 04 16:45:43.139999 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:45:43.139936 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0\": container with ID starting with 03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0 not found: ID does not exist" containerID="03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0" Aug 04 16:45:43.139999 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.139972 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0"} err="failed to get container status \"03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0\": rpc error: code = NotFound desc = could not find container \"03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0\": container with ID starting with 03fc52f1ca1f85ad5a374744b2adbb20c9e60516578f49aafa800fba913c07d0 not found: ID does not exist" Aug 04 16:45:43.142115 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.142088 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e13060-openshift-default-7c78484864-d2hsz"] Aug 04 16:45:43.239794 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.239761 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8"] Aug 04 16:45:43.240806 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:45:43.240769 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd910e5c0_7f63_4a32_9956_ab6019773273.slice/crio-537f24892e72b77151354f416c5ebc9e8c83ff69c19bfda49a04e6cfc2ca5fff WatchSource:0}: Error finding container 537f24892e72b77151354f416c5ebc9e8c83ff69c19bfda49a04e6cfc2ca5fff: Status 404 returned error can't find the container with id 537f24892e72b77151354f416c5ebc9e8c83ff69c19bfda49a04e6cfc2ca5fff Aug 04 16:45:43.264711 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.264688 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28"] Aug 04 16:45:43.272001 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:45:43.271955 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e2aa30_834e_4578_a6b3_239e3456f69f.slice/crio-211d32eb81d992be91bacdcbea44d5d0d4141a9a7faf105f79b811d84ed73de7 WatchSource:0}: Error finding container 211d32eb81d992be91bacdcbea44d5d0d4141a9a7faf105f79b811d84ed73de7: Status 404 returned error can't find the container with id 211d32eb81d992be91bacdcbea44d5d0d4141a9a7faf105f79b811d84ed73de7 Aug 04 16:45:43.274338 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:43.274310 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02655b32-4a3f-4783-9f99-c725e23bf8f1" path="/var/lib/kubelet/pods/02655b32-4a3f-4783-9f99-c725e23bf8f1/volumes" Aug 04 16:45:44.095242 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:44.095207 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" event={"ID":"d910e5c0-7f63-4a32-9956-ab6019773273","Type":"ContainerStarted","Data":"73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895"} Aug 04 16:45:44.095454 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:44.095280 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:44.095454 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:44.095302 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" event={"ID":"d910e5c0-7f63-4a32-9956-ab6019773273","Type":"ContainerStarted","Data":"537f24892e72b77151354f416c5ebc9e8c83ff69c19bfda49a04e6cfc2ca5fff"} Aug 04 16:45:44.096953 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:44.096927 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" event={"ID":"37e2aa30-834e-4578-a6b3-239e3456f69f","Type":"ContainerStarted","Data":"637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860"} Aug 04 16:45:44.097079 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:44.096967 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" event={"ID":"37e2aa30-834e-4578-a6b3-239e3456f69f","Type":"ContainerStarted","Data":"211d32eb81d992be91bacdcbea44d5d0d4141a9a7faf105f79b811d84ed73de7"} Aug 04 16:45:44.097079 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:44.097029 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:44.110741 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:44.110671 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" podStartSLOduration=2.110656654 podStartE2EDuration="2.110656654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:45:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:45:44.109912766 +0000 UTC m=+2973.432336247" watchObservedRunningTime="2026-08-04 16:45:44.110656654 +0000 UTC m=+2973.433080133" Aug 04 16:45:44.127310 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:44.127255 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" podStartSLOduration=2.127239269 podStartE2EDuration="2.127239269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:45:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:45:44.125653192 +0000 UTC m=+2973.448076672" watchObservedRunningTime="2026-08-04 16:45:44.127239269 +0000 UTC m=+2973.449662749" Aug 04 16:45:55.070558 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.070522 2556 patch_prober.go:28] interesting pod/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Aug 04 16:45:55.070994 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.070587 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Aug 04 16:45:55.105430 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.105353 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:45:55.105430 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.105423 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:45:55.159832 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.159796 2556 generic.go:356] "Generic (PLEG): container finished" podID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerID="52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461" exitCode=0 Aug 04 16:45:55.159992 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.159861 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" event={"ID":"acfbe88c-858a-47f4-a927-c69fe0fedb55","Type":"ContainerDied","Data":"52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461"} Aug 04 16:45:55.160302 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.160287 2556 scope.go:117] "RemoveContainer" containerID="52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461" Aug 04 16:45:55.161381 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.161354 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:45:55.161485 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.161435 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:45:55.161554 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:55.161485 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:45:56.167013 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:56.166973 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" event={"ID":"acfbe88c-858a-47f4-a927-c69fe0fedb55","Type":"ContainerStarted","Data":"8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430"} Aug 04 16:45:56.235144 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:56.235101 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:45:56.235365 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:56.235340 2556 patch_prober.go:28] interesting pod/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Aug 04 16:45:56.235417 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:56.235397 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Aug 04 16:45:57.235936 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:57.235894 2556 patch_prober.go:28] interesting pod/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Aug 04 16:45:57.236360 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:57.235957 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Aug 04 16:45:58.236183 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:58.236143 2556 patch_prober.go:28] interesting pod/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Aug 04 16:45:58.236601 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:58.236202 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Aug 04 16:45:59.235886 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:59.235840 2556 patch_prober.go:28] interesting pod/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Aug 04 16:45:59.236107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:45:59.235919 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Aug 04 16:46:00.235431 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:00.235390 2556 patch_prober.go:28] interesting pod/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Aug 04 16:46:00.235879 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:00.235448 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Aug 04 16:46:01.239384 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:01.239348 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:46:01.239809 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:01.239644 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:46:01.240412 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:01.240391 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:46:06.664138 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.664100 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7"] Aug 04 16:46:06.668425 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.668402 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.671420 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.671394 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-5a3cb8-openshift-default-dockercfg-hjlzq\"" Aug 04 16:46:06.681970 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.681943 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7"] Aug 04 16:46:06.807172 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.807138 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf925\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-kube-api-access-rf925\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.807369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.807177 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-token\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.807369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.807217 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-certs\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.807369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.807241 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-credential-socket\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.807369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.807259 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istiod-ca-cert\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.807369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.807278 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-data\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.807573 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.807382 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-podinfo\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.807573 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.807414 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-envoy\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.807573 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.807440 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-socket\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909032 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.908992 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-certs\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909032 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909035 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-credential-socket\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909300 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istiod-ca-cert\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909300 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909193 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-data\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909300 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909292 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-podinfo\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909327 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-envoy\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909368 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-socket\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909407 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-certs\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909462 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909451 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf925\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-kube-api-access-rf925\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909490 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-token\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909451 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-credential-socket\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909522 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-data\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909662 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-socket\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.909917 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.909802 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istiod-ca-cert\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.911771 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.911743 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-envoy\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.911907 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.911888 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-podinfo\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.920204 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.920129 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-token\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.920344 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.920323 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf925\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-kube-api-access-rf925\") pod \"e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:06.980178 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:06.980140 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:07.124087 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:07.124060 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7"] Aug 04 16:46:07.125647 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:46:07.125620 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f78e2e_bdfe_44ad_83ae_4e7ac5a27c1b.slice/crio-3c28fb273de3f0a6c05234417d803c0212791ec5c03d63339d4c16cde3aee0d1 WatchSource:0}: Error finding container 3c28fb273de3f0a6c05234417d803c0212791ec5c03d63339d4c16cde3aee0d1: Status 404 returned error can't find the container with id 3c28fb273de3f0a6c05234417d803c0212791ec5c03d63339d4c16cde3aee0d1 Aug 04 16:46:07.128064 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:07.128020 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:46:07.128152 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:07.128121 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:46:07.128193 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:07.128162 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:46:07.249362 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:07.249325 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" event={"ID":"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b","Type":"ContainerStarted","Data":"0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2"} Aug 04 16:46:07.249576 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:07.249393 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" event={"ID":"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b","Type":"ContainerStarted","Data":"3c28fb273de3f0a6c05234417d803c0212791ec5c03d63339d4c16cde3aee0d1"} Aug 04 16:46:07.269224 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:07.269162 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" podStartSLOduration=1.269143986 podStartE2EDuration="1.269143986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:46:07.267576472 +0000 UTC m=+2996.589999952" watchObservedRunningTime="2026-08-04 16:46:07.269143986 +0000 UTC m=+2996.591567465" Aug 04 16:46:07.980516 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:07.980486 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:08.986194 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:08.986163 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:09.260572 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:09.260534 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:09.261552 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:09.261531 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:11.132003 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.131965 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9"] Aug 04 16:46:11.135911 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.135891 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:11.138556 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.138530 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-5a3cb8\"" Aug 04 16:46:11.145510 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.145483 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9"] Aug 04 16:46:11.152323 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.152291 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9\" (UID: \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:11.152490 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.152353 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq7tf\" (UniqueName: \"kubernetes.io/projected/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-kube-api-access-pq7tf\") pod \"payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9\" (UID: \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:11.179400 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.179368 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr"] Aug 04 16:46:11.183164 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.183140 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:11.185754 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.185715 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-5a3cb8-dockercfg-g8scv\"" Aug 04 16:46:11.193857 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.193822 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr"] Aug 04 16:46:11.254565 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.254171 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89428bbc-eb8a-45bf-9299-35e446d987ed-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr\" (UID: \"89428bbc-eb8a-45bf-9299-35e446d987ed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:11.254565 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.254321 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9\" (UID: \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:11.254565 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.254363 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgvsh\" (UniqueName: \"kubernetes.io/projected/89428bbc-eb8a-45bf-9299-35e446d987ed-kube-api-access-pgvsh\") pod \"payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr\" (UID: \"89428bbc-eb8a-45bf-9299-35e446d987ed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:11.254565 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.254422 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq7tf\" (UniqueName: \"kubernetes.io/projected/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-kube-api-access-pq7tf\") pod \"payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9\" (UID: \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:11.255845 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.255677 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9\" (UID: \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:11.264366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.264336 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq7tf\" (UniqueName: \"kubernetes.io/projected/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-kube-api-access-pq7tf\") pod \"payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9\" (UID: \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:11.355063 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.355022 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89428bbc-eb8a-45bf-9299-35e446d987ed-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr\" (UID: \"89428bbc-eb8a-45bf-9299-35e446d987ed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:11.355282 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.355201 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgvsh\" (UniqueName: \"kubernetes.io/projected/89428bbc-eb8a-45bf-9299-35e446d987ed-kube-api-access-pgvsh\") pod \"payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr\" (UID: \"89428bbc-eb8a-45bf-9299-35e446d987ed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:11.355907 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.355885 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89428bbc-eb8a-45bf-9299-35e446d987ed-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr\" (UID: \"89428bbc-eb8a-45bf-9299-35e446d987ed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:11.363699 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.363675 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgvsh\" (UniqueName: \"kubernetes.io/projected/89428bbc-eb8a-45bf-9299-35e446d987ed-kube-api-access-pgvsh\") pod \"payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr\" (UID: \"89428bbc-eb8a-45bf-9299-35e446d987ed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:11.448707 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.448605 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:11.495189 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.495144 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:11.615524 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.615488 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9"] Aug 04 16:46:11.620474 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:46:11.620428 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c8d539_1e12_436f_94ea_e7a7b2a37f73.slice/crio-6e5206abdf419387e2bfe8b31e654dac00e8ec90cb1eea89b2ab9d858e73065a WatchSource:0}: Error finding container 6e5206abdf419387e2bfe8b31e654dac00e8ec90cb1eea89b2ab9d858e73065a: Status 404 returned error can't find the container with id 6e5206abdf419387e2bfe8b31e654dac00e8ec90cb1eea89b2ab9d858e73065a Aug 04 16:46:11.673918 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:11.673893 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr"] Aug 04 16:46:11.677217 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:46:11.677181 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89428bbc_eb8a_45bf_9299_35e446d987ed.slice/crio-326548c1315bc55245c83cd74cf7059ad819543f8104ca8c038a648951cd013a WatchSource:0}: Error finding container 326548c1315bc55245c83cd74cf7059ad819543f8104ca8c038a648951cd013a: Status 404 returned error can't find the container with id 326548c1315bc55245c83cd74cf7059ad819543f8104ca8c038a648951cd013a Aug 04 16:46:12.288390 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:12.288353 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" event={"ID":"89428bbc-eb8a-45bf-9299-35e446d987ed","Type":"ContainerStarted","Data":"ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4"} Aug 04 16:46:12.288941 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:12.288414 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" event={"ID":"89428bbc-eb8a-45bf-9299-35e446d987ed","Type":"ContainerStarted","Data":"326548c1315bc55245c83cd74cf7059ad819543f8104ca8c038a648951cd013a"} Aug 04 16:46:12.288941 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:12.288443 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:12.290614 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:12.290583 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" event={"ID":"f7c8d539-1e12-436f-94ea-e7a7b2a37f73","Type":"ContainerStarted","Data":"bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1"} Aug 04 16:46:12.290792 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:12.290642 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" event={"ID":"f7c8d539-1e12-436f-94ea-e7a7b2a37f73","Type":"ContainerStarted","Data":"6e5206abdf419387e2bfe8b31e654dac00e8ec90cb1eea89b2ab9d858e73065a"} Aug 04 16:46:12.290792 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:12.290670 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:12.308032 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:12.307975 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" podStartSLOduration=1.307958183 podStartE2EDuration="1.307958183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:46:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:46:12.306231132 +0000 UTC m=+3001.628654612" watchObservedRunningTime="2026-08-04 16:46:12.307958183 +0000 UTC m=+3001.630381640" Aug 04 16:46:12.323499 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:12.323441 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" podStartSLOduration=1.323424879 podStartE2EDuration="1.323424879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:46:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:46:12.322304517 +0000 UTC m=+3001.644728002" watchObservedRunningTime="2026-08-04 16:46:12.323424879 +0000 UTC m=+3001.645848360" Aug 04 16:46:23.299400 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:23.299368 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:46:23.299888 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:23.299451 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:46:23.371647 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:23.371610 2556 generic.go:356] "Generic (PLEG): container finished" podID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerID="0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2" exitCode=0 Aug 04 16:46:23.371860 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:23.371684 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" event={"ID":"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b","Type":"ContainerDied","Data":"0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2"} Aug 04 16:46:23.372241 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:23.372223 2556 scope.go:117] "RemoveContainer" containerID="0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2" Aug 04 16:46:23.373380 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:23.373347 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:46:23.373475 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:23.373419 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:46:23.373475 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:23.373450 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 16:46:24.261505 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:24.261461 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:24.381926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:24.381881 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" event={"ID":"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b","Type":"ContainerStarted","Data":"9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a"} Aug 04 16:46:24.982231 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:24.982181 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:24.982432 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:24.982362 2556 patch_prober.go:28] interesting pod/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Aug 04 16:46:24.982525 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:24.982426 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Aug 04 16:46:25.981483 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:25.981442 2556 patch_prober.go:28] interesting pod/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Aug 04 16:46:25.982478 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:25.982065 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Aug 04 16:46:26.981242 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:26.981200 2556 patch_prober.go:28] interesting pod/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Aug 04 16:46:26.981565 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:26.981532 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Aug 04 16:46:27.981645 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:27.981597 2556 patch_prober.go:28] interesting pod/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Aug 04 16:46:27.982165 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:27.981681 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Aug 04 16:46:28.980663 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:28.980614 2556 patch_prober.go:28] interesting pod/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Aug 04 16:46:28.980924 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:28.980690 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Aug 04 16:46:29.984865 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:29.984830 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:29.985264 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:29.985122 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:46:29.985881 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:46:29.985865 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 16:55:17.311416 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.311373 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz"] Aug 04 16:55:17.315168 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.315143 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:17.321040 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.321014 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-bbe0dc-dockercfg-smqhl\"" Aug 04 16:55:17.321256 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.321241 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-bbe0dc\"" Aug 04 16:55:17.328420 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.328395 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb"] Aug 04 16:55:17.332276 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.332252 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:17.333802 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.333763 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz"] Aug 04 16:55:17.343747 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.343707 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb"] Aug 04 16:55:17.405396 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.405355 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42kd2\" (UniqueName: \"kubernetes.io/projected/2a1266ad-3295-4bae-89b5-735d889f72da-kube-api-access-42kd2\") pod \"payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz\" (UID: \"2a1266ad-3295-4bae-89b5-735d889f72da\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:17.405567 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.405412 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f298c5e5-00fa-470d-9942-f9fe63daa204-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb\" (UID: \"f298c5e5-00fa-470d-9942-f9fe63daa204\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:17.405567 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.405471 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gcc6\" (UniqueName: \"kubernetes.io/projected/f298c5e5-00fa-470d-9942-f9fe63daa204-kube-api-access-6gcc6\") pod \"payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb\" (UID: \"f298c5e5-00fa-470d-9942-f9fe63daa204\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:17.405647 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.405574 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1266ad-3295-4bae-89b5-735d889f72da-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz\" (UID: \"2a1266ad-3295-4bae-89b5-735d889f72da\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:17.506291 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.506251 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1266ad-3295-4bae-89b5-735d889f72da-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz\" (UID: \"2a1266ad-3295-4bae-89b5-735d889f72da\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:17.506486 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.506328 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42kd2\" (UniqueName: \"kubernetes.io/projected/2a1266ad-3295-4bae-89b5-735d889f72da-kube-api-access-42kd2\") pod \"payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz\" (UID: \"2a1266ad-3295-4bae-89b5-735d889f72da\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:17.506486 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.506374 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f298c5e5-00fa-470d-9942-f9fe63daa204-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb\" (UID: \"f298c5e5-00fa-470d-9942-f9fe63daa204\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:17.506486 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.506412 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gcc6\" (UniqueName: \"kubernetes.io/projected/f298c5e5-00fa-470d-9942-f9fe63daa204-kube-api-access-6gcc6\") pod \"payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb\" (UID: \"f298c5e5-00fa-470d-9942-f9fe63daa204\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:17.507104 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.507081 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1266ad-3295-4bae-89b5-735d889f72da-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz\" (UID: \"2a1266ad-3295-4bae-89b5-735d889f72da\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:17.507275 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.507254 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f298c5e5-00fa-470d-9942-f9fe63daa204-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb\" (UID: \"f298c5e5-00fa-470d-9942-f9fe63daa204\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:17.514708 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.514679 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42kd2\" (UniqueName: \"kubernetes.io/projected/2a1266ad-3295-4bae-89b5-735d889f72da-kube-api-access-42kd2\") pod \"payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz\" (UID: \"2a1266ad-3295-4bae-89b5-735d889f72da\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:17.514708 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.514700 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gcc6\" (UniqueName: \"kubernetes.io/projected/f298c5e5-00fa-470d-9942-f9fe63daa204-kube-api-access-6gcc6\") pod \"payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb\" (UID: \"f298c5e5-00fa-470d-9942-f9fe63daa204\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:17.625604 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.625508 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:17.644848 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.644713 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:17.821949 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.821919 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb"] Aug 04 16:55:17.823488 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:55:17.823458 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf298c5e5_00fa_470d_9942_f9fe63daa204.slice/crio-83c0f12453ef8ddcd5dd6524d6650a6c0329f26834da1a384607b1edc6216b39 WatchSource:0}: Error finding container 83c0f12453ef8ddcd5dd6524d6650a6c0329f26834da1a384607b1edc6216b39: Status 404 returned error can't find the container with id 83c0f12453ef8ddcd5dd6524d6650a6c0329f26834da1a384607b1edc6216b39 Aug 04 16:55:17.991122 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:17.991093 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz"] Aug 04 16:55:17.993319 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:55:17.993278 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a1266ad_3295_4bae_89b5_735d889f72da.slice/crio-8b8a9c78b74b966898f63bbaf11ea7f9ae8b33e0cd2510be7bc57b8813e13ef7 WatchSource:0}: Error finding container 8b8a9c78b74b966898f63bbaf11ea7f9ae8b33e0cd2510be7bc57b8813e13ef7: Status 404 returned error can't find the container with id 8b8a9c78b74b966898f63bbaf11ea7f9ae8b33e0cd2510be7bc57b8813e13ef7 Aug 04 16:55:18.263393 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:18.263356 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" event={"ID":"f298c5e5-00fa-470d-9942-f9fe63daa204","Type":"ContainerStarted","Data":"5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243"} Aug 04 16:55:18.263587 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:18.263422 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:18.263587 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:18.263448 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" event={"ID":"f298c5e5-00fa-470d-9942-f9fe63daa204","Type":"ContainerStarted","Data":"83c0f12453ef8ddcd5dd6524d6650a6c0329f26834da1a384607b1edc6216b39"} Aug 04 16:55:18.264971 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:18.264926 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" event={"ID":"2a1266ad-3295-4bae-89b5-735d889f72da","Type":"ContainerStarted","Data":"efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23"} Aug 04 16:55:18.265109 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:18.264979 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" event={"ID":"2a1266ad-3295-4bae-89b5-735d889f72da","Type":"ContainerStarted","Data":"8b8a9c78b74b966898f63bbaf11ea7f9ae8b33e0cd2510be7bc57b8813e13ef7"} Aug 04 16:55:18.265109 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:18.265008 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:18.280636 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:18.280587 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" podStartSLOduration=1.280567293 podStartE2EDuration="1.280567293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:55:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:55:18.279663699 +0000 UTC m=+3547.602087196" watchObservedRunningTime="2026-08-04 16:55:18.280567293 +0000 UTC m=+3547.602990780" Aug 04 16:55:18.296635 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:18.296578 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" podStartSLOduration=1.296557578 podStartE2EDuration="1.296557578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:55:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:55:18.295940727 +0000 UTC m=+3547.618364219" watchObservedRunningTime="2026-08-04 16:55:18.296557578 +0000 UTC m=+3547.618981059" Aug 04 16:55:29.273610 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:29.273574 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:55:29.274049 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:29.273851 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:55:46.630566 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.630526 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6"] Aug 04 16:55:46.634334 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.634306 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:55:46.636910 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.636882 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-54f057\"" Aug 04 16:55:46.637211 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.637192 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-54f057-dockercfg-ws7hj\"" Aug 04 16:55:46.645766 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.645740 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6"] Aug 04 16:55:46.704648 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.704616 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws282\" (UniqueName: \"kubernetes.io/projected/dcb65243-9127-430b-aefa-8a83de79d5a7-kube-api-access-ws282\") pod \"payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6\" (UID: \"dcb65243-9127-430b-aefa-8a83de79d5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:55:46.704863 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.704799 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb65243-9127-430b-aefa-8a83de79d5a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6\" (UID: \"dcb65243-9127-430b-aefa-8a83de79d5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:55:46.805543 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.805502 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb65243-9127-430b-aefa-8a83de79d5a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6\" (UID: \"dcb65243-9127-430b-aefa-8a83de79d5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:55:46.805800 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.805579 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws282\" (UniqueName: \"kubernetes.io/projected/dcb65243-9127-430b-aefa-8a83de79d5a7-kube-api-access-ws282\") pod \"payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6\" (UID: \"dcb65243-9127-430b-aefa-8a83de79d5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:55:46.806342 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.806313 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb65243-9127-430b-aefa-8a83de79d5a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6\" (UID: \"dcb65243-9127-430b-aefa-8a83de79d5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:55:46.815545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.815509 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws282\" (UniqueName: \"kubernetes.io/projected/dcb65243-9127-430b-aefa-8a83de79d5a7-kube-api-access-ws282\") pod \"payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6\" (UID: \"dcb65243-9127-430b-aefa-8a83de79d5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:55:46.947503 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:46.947409 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:55:47.309828 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:47.309793 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6"] Aug 04 16:55:47.310984 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:55:47.310941 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb65243_9127_430b_aefa_8a83de79d5a7.slice/crio-2a6082651f291f1454856719ecfeaa381cde4703c6fcd462df1522fdbea2a481 WatchSource:0}: Error finding container 2a6082651f291f1454856719ecfeaa381cde4703c6fcd462df1522fdbea2a481: Status 404 returned error can't find the container with id 2a6082651f291f1454856719ecfeaa381cde4703c6fcd462df1522fdbea2a481 Aug 04 16:55:47.428131 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:47.428093 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" event={"ID":"dcb65243-9127-430b-aefa-8a83de79d5a7","Type":"ContainerStarted","Data":"398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995"} Aug 04 16:55:47.428327 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:47.428158 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" event={"ID":"dcb65243-9127-430b-aefa-8a83de79d5a7","Type":"ContainerStarted","Data":"2a6082651f291f1454856719ecfeaa381cde4703c6fcd462df1522fdbea2a481"} Aug 04 16:55:47.428327 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:47.428189 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:55:47.446120 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:47.446060 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" podStartSLOduration=1.446042901 podStartE2EDuration="1.446042901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:55:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:55:47.443515404 +0000 UTC m=+3576.765938885" watchObservedRunningTime="2026-08-04 16:55:47.446042901 +0000 UTC m=+3576.768466381" Aug 04 16:55:58.436517 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:55:58.436485 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:56:16.776480 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:56:16.776452 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6vftr_494ec520-130e-4ca2-a9ca-4f11a19f69eb/payload-processing/0.log" Aug 04 16:57:17.185937 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:17.185905 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb_f298c5e5-00fa-470d-9942-f9fe63daa204/payload-processing/0.log" Aug 04 16:57:17.324448 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:17.324416 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6vftr_494ec520-130e-4ca2-a9ca-4f11a19f69eb/payload-processing/0.log" Aug 04 16:57:23.236713 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.236678 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r"] Aug 04 16:57:23.240535 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.240519 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:23.243134 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.243108 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-4dd86f-dockercfg-szs2t\"" Aug 04 16:57:23.243263 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.243138 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-4dd86f\"" Aug 04 16:57:23.250355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.250326 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r"] Aug 04 16:57:23.275393 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.275363 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv"] Aug 04 16:57:23.279030 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.279011 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:23.285107 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.285075 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv"] Aug 04 16:57:23.387319 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.387272 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r\" (UID: \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:23.387557 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.387329 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd085df-0cf2-4471-b205-eff5aead06fa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv\" (UID: \"7bd085df-0cf2-4471-b205-eff5aead06fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:23.387557 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.387385 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9gqc\" (UniqueName: \"kubernetes.io/projected/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-kube-api-access-m9gqc\") pod \"payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r\" (UID: \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:23.387557 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.387468 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmh6\" (UniqueName: \"kubernetes.io/projected/7bd085df-0cf2-4471-b205-eff5aead06fa-kube-api-access-cbmh6\") pod \"payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv\" (UID: \"7bd085df-0cf2-4471-b205-eff5aead06fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:23.488309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.488224 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r\" (UID: \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:23.488309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.488259 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd085df-0cf2-4471-b205-eff5aead06fa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv\" (UID: \"7bd085df-0cf2-4471-b205-eff5aead06fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:23.488309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.488286 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9gqc\" (UniqueName: \"kubernetes.io/projected/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-kube-api-access-m9gqc\") pod \"payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r\" (UID: \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:23.488309 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.488308 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmh6\" (UniqueName: \"kubernetes.io/projected/7bd085df-0cf2-4471-b205-eff5aead06fa-kube-api-access-cbmh6\") pod \"payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv\" (UID: \"7bd085df-0cf2-4471-b205-eff5aead06fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:23.489253 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.489215 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r\" (UID: \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:23.489365 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.489215 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd085df-0cf2-4471-b205-eff5aead06fa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv\" (UID: \"7bd085df-0cf2-4471-b205-eff5aead06fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:23.496481 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.496449 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9gqc\" (UniqueName: \"kubernetes.io/projected/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-kube-api-access-m9gqc\") pod \"payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r\" (UID: \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:23.497007 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.496986 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmh6\" (UniqueName: \"kubernetes.io/projected/7bd085df-0cf2-4471-b205-eff5aead06fa-kube-api-access-cbmh6\") pod \"payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv\" (UID: \"7bd085df-0cf2-4471-b205-eff5aead06fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:23.552320 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.552284 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:23.590487 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.590459 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:23.713283 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.711679 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r"] Aug 04 16:57:23.715694 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:57:23.715656 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b0c7d2_cc85_4d69_ab6a_4eb7bf36c549.slice/crio-46ce2572dc183d57202c9f7b42aacf9d8c473d7b9720e539d713ebca5c9fc923 WatchSource:0}: Error finding container 46ce2572dc183d57202c9f7b42aacf9d8c473d7b9720e539d713ebca5c9fc923: Status 404 returned error can't find the container with id 46ce2572dc183d57202c9f7b42aacf9d8c473d7b9720e539d713ebca5c9fc923 Aug 04 16:57:23.958193 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.958148 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" event={"ID":"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549","Type":"ContainerStarted","Data":"027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2"} Aug 04 16:57:23.958373 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.958218 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" event={"ID":"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549","Type":"ContainerStarted","Data":"46ce2572dc183d57202c9f7b42aacf9d8c473d7b9720e539d713ebca5c9fc923"} Aug 04 16:57:23.958373 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.958320 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:23.968124 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.968101 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv"] Aug 04 16:57:23.969442 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:57:23.969403 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd085df_0cf2_4471_b205_eff5aead06fa.slice/crio-7558617173d84a081e5539c59110234661c1cfdffcc0c3a2b8518042fcc68f22 WatchSource:0}: Error finding container 7558617173d84a081e5539c59110234661c1cfdffcc0c3a2b8518042fcc68f22: Status 404 returned error can't find the container with id 7558617173d84a081e5539c59110234661c1cfdffcc0c3a2b8518042fcc68f22 Aug 04 16:57:23.975255 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:23.975211 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" podStartSLOduration=0.975195579 podStartE2EDuration="975.195579ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:57:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:57:23.9743929 +0000 UTC m=+3673.296816381" watchObservedRunningTime="2026-08-04 16:57:23.975195579 +0000 UTC m=+3673.297619059" Aug 04 16:57:24.965405 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:24.965369 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" event={"ID":"7bd085df-0cf2-4471-b205-eff5aead06fa","Type":"ContainerStarted","Data":"722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8"} Aug 04 16:57:24.965874 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:24.965428 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" event={"ID":"7bd085df-0cf2-4471-b205-eff5aead06fa","Type":"ContainerStarted","Data":"7558617173d84a081e5539c59110234661c1cfdffcc0c3a2b8518042fcc68f22"} Aug 04 16:57:24.965874 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:24.965456 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:24.987297 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:24.987251 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" podStartSLOduration=1.98723297 podStartE2EDuration="1.98723297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:57:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:57:24.985313474 +0000 UTC m=+3674.307736980" watchObservedRunningTime="2026-08-04 16:57:24.98723297 +0000 UTC m=+3674.309656450" Aug 04 16:57:34.967546 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:34.967456 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:35.973701 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:35.973664 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:53.299987 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.299950 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv"] Aug 04 16:57:53.300510 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.300216 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" podUID="7bd085df-0cf2-4471-b205-eff5aead06fa" containerName="payload-processing" containerID="cri-o://722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8" gracePeriod=30 Aug 04 16:57:53.307032 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.307005 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r"] Aug 04 16:57:53.308698 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.307298 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" podUID="28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549" containerName="payload-pre-processing" containerID="cri-o://027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2" gracePeriod=30 Aug 04 16:57:53.600864 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.600833 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:53.603003 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.602983 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:53.681371 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.681336 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbmh6\" (UniqueName: \"kubernetes.io/projected/7bd085df-0cf2-4471-b205-eff5aead06fa-kube-api-access-cbmh6\") pod \"7bd085df-0cf2-4471-b205-eff5aead06fa\" (UID: \"7bd085df-0cf2-4471-b205-eff5aead06fa\") " Aug 04 16:57:53.681542 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.681453 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9gqc\" (UniqueName: \"kubernetes.io/projected/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-kube-api-access-m9gqc\") pod \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\" (UID: \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\") " Aug 04 16:57:53.681542 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.681481 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-plugins-config-volume\") pod \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\" (UID: \"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549\") " Aug 04 16:57:53.681542 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.681499 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd085df-0cf2-4471-b205-eff5aead06fa-plugins-config-volume\") pod \"7bd085df-0cf2-4471-b205-eff5aead06fa\" (UID: \"7bd085df-0cf2-4471-b205-eff5aead06fa\") " Aug 04 16:57:53.681951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.681923 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549" (UID: "28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:57:53.681951 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.681939 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bd085df-0cf2-4471-b205-eff5aead06fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7bd085df-0cf2-4471-b205-eff5aead06fa" (UID: "7bd085df-0cf2-4471-b205-eff5aead06fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:57:53.683566 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.683545 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bd085df-0cf2-4471-b205-eff5aead06fa-kube-api-access-cbmh6" (OuterVolumeSpecName: "kube-api-access-cbmh6") pod "7bd085df-0cf2-4471-b205-eff5aead06fa" (UID: "7bd085df-0cf2-4471-b205-eff5aead06fa"). InnerVolumeSpecName "kube-api-access-cbmh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:57:53.683566 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.683561 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-kube-api-access-m9gqc" (OuterVolumeSpecName: "kube-api-access-m9gqc") pod "28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549" (UID: "28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549"). InnerVolumeSpecName "kube-api-access-m9gqc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:57:53.783088 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.783049 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9gqc\" (UniqueName: \"kubernetes.io/projected/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-kube-api-access-m9gqc\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:57:53.783088 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.783081 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:57:53.783088 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.783092 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd085df-0cf2-4471-b205-eff5aead06fa-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:57:53.783330 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:53.783100 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbmh6\" (UniqueName: \"kubernetes.io/projected/7bd085df-0cf2-4471-b205-eff5aead06fa-kube-api-access-cbmh6\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:57:54.130940 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.130904 2556 generic.go:356] "Generic (PLEG): container finished" podID="7bd085df-0cf2-4471-b205-eff5aead06fa" containerID="722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8" exitCode=0 Aug 04 16:57:54.131129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.130953 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" Aug 04 16:57:54.131129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.130991 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" event={"ID":"7bd085df-0cf2-4471-b205-eff5aead06fa","Type":"ContainerDied","Data":"722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8"} Aug 04 16:57:54.131129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.131046 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv" event={"ID":"7bd085df-0cf2-4471-b205-eff5aead06fa","Type":"ContainerDied","Data":"7558617173d84a081e5539c59110234661c1cfdffcc0c3a2b8518042fcc68f22"} Aug 04 16:57:54.131129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.131076 2556 scope.go:117] "RemoveContainer" containerID="722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8" Aug 04 16:57:54.132346 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.132322 2556 generic.go:356] "Generic (PLEG): container finished" podID="28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549" containerID="027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2" exitCode=0 Aug 04 16:57:54.132461 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.132392 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" Aug 04 16:57:54.132461 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.132426 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" event={"ID":"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549","Type":"ContainerDied","Data":"027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2"} Aug 04 16:57:54.132542 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.132479 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r" event={"ID":"28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549","Type":"ContainerDied","Data":"46ce2572dc183d57202c9f7b42aacf9d8c473d7b9720e539d713ebca5c9fc923"} Aug 04 16:57:54.140674 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.140654 2556 scope.go:117] "RemoveContainer" containerID="722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8" Aug 04 16:57:54.141013 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:57:54.140989 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8\": container with ID starting with 722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8 not found: ID does not exist" containerID="722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8" Aug 04 16:57:54.141082 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.141021 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8"} err="failed to get container status \"722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8\": rpc error: code = NotFound desc = could not find container \"722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8\": container with ID starting with 722c77ba210988719023c2dbead4496ab0f62f55646267822f86486b2ac015b8 not found: ID does not exist" Aug 04 16:57:54.141082 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.141037 2556 scope.go:117] "RemoveContainer" containerID="027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2" Aug 04 16:57:54.149895 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.149872 2556 scope.go:117] "RemoveContainer" containerID="027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2" Aug 04 16:57:54.150172 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:57:54.150152 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2\": container with ID starting with 027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2 not found: ID does not exist" containerID="027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2" Aug 04 16:57:54.150227 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.150180 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2"} err="failed to get container status \"027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2\": rpc error: code = NotFound desc = could not find container \"027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2\": container with ID starting with 027604dab5d79dfe1b57e2069c6eb868be9cd10f17e4f8de59303ca01e0c12f2 not found: ID does not exist" Aug 04 16:57:54.161355 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.161328 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv"] Aug 04 16:57:54.165901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.165878 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4dd86f-7cfddd8d76-nvlsv"] Aug 04 16:57:54.178129 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.178106 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r"] Aug 04 16:57:54.184425 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:54.184399 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4dd86f-7b45675fc6-p8f5r"] Aug 04 16:57:55.274171 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:55.274140 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549" path="/var/lib/kubelet/pods/28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549/volumes" Aug 04 16:57:55.274549 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:57:55.274504 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bd085df-0cf2-4471-b205-eff5aead06fa" path="/var/lib/kubelet/pods/7bd085df-0cf2-4471-b205-eff5aead06fa/volumes" Aug 04 16:58:03.844269 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.844229 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn"] Aug 04 16:58:03.844693 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.844680 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549" containerName="payload-pre-processing" Aug 04 16:58:03.844767 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.844694 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549" containerName="payload-pre-processing" Aug 04 16:58:03.844767 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.844717 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd085df-0cf2-4471-b205-eff5aead06fa" containerName="payload-processing" Aug 04 16:58:03.844767 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.844739 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd085df-0cf2-4471-b205-eff5aead06fa" containerName="payload-processing" Aug 04 16:58:03.844879 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.844801 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="28b0c7d2-cc85-4d69-ab6a-4eb7bf36c549" containerName="payload-pre-processing" Aug 04 16:58:03.844925 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.844913 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bd085df-0cf2-4471-b205-eff5aead06fa" containerName="payload-processing" Aug 04 16:58:03.848269 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.848250 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" Aug 04 16:58:03.854189 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.854162 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-psz5k\"" Aug 04 16:58:03.859231 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.859210 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn"] Aug 04 16:58:03.980929 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:03.980890 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8f63d2-8f98-4711-b570-1ce21a150c73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn\" (UID: \"fa8f63d2-8f98-4711-b570-1ce21a150c73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" Aug 04 16:58:04.082493 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:04.082456 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8f63d2-8f98-4711-b570-1ce21a150c73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn\" (UID: \"fa8f63d2-8f98-4711-b570-1ce21a150c73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" Aug 04 16:58:04.083139 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:04.083114 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8f63d2-8f98-4711-b570-1ce21a150c73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn\" (UID: \"fa8f63d2-8f98-4711-b570-1ce21a150c73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" Aug 04 16:58:04.160341 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:04.160226 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" Aug 04 16:58:04.319975 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:04.319947 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn"] Aug 04 16:58:04.321817 ip-10-0-132-227 kubenswrapper[2556]: W0804 16:58:04.321789 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8f63d2_8f98_4711_b570_1ce21a150c73.slice/crio-5d2c808170e9cd6685381cd65ababbfb4a0949d87ccca298f5a680eb19ea2fc9 WatchSource:0}: Error finding container 5d2c808170e9cd6685381cd65ababbfb4a0949d87ccca298f5a680eb19ea2fc9: Status 404 returned error can't find the container with id 5d2c808170e9cd6685381cd65ababbfb4a0949d87ccca298f5a680eb19ea2fc9 Aug 04 16:58:05.205397 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:05.205358 2556 generic.go:356] "Generic (PLEG): container finished" podID="fa8f63d2-8f98-4711-b570-1ce21a150c73" containerID="70515b0ab8971fa03e0499d37b5567ede3c1284837e111f95b07b170ad46fd14" exitCode=0 Aug 04 16:58:05.205781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:05.205428 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" event={"ID":"fa8f63d2-8f98-4711-b570-1ce21a150c73","Type":"ContainerDied","Data":"70515b0ab8971fa03e0499d37b5567ede3c1284837e111f95b07b170ad46fd14"} Aug 04 16:58:05.205781 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:05.205473 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" event={"ID":"fa8f63d2-8f98-4711-b570-1ce21a150c73","Type":"ContainerStarted","Data":"5d2c808170e9cd6685381cd65ababbfb4a0949d87ccca298f5a680eb19ea2fc9"} Aug 04 16:58:06.353594 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:06.353569 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" Aug 04 16:58:06.407612 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:06.407577 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8f63d2-8f98-4711-b570-1ce21a150c73-maas-api-service-ca\") pod \"fa8f63d2-8f98-4711-b570-1ce21a150c73\" (UID: \"fa8f63d2-8f98-4711-b570-1ce21a150c73\") " Aug 04 16:58:06.407962 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:06.407939 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa8f63d2-8f98-4711-b570-1ce21a150c73-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa8f63d2-8f98-4711-b570-1ce21a150c73" (UID: "fa8f63d2-8f98-4711-b570-1ce21a150c73"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:58:06.508516 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:06.508482 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8f63d2-8f98-4711-b570-1ce21a150c73-maas-api-service-ca\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:58:07.217443 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:07.217409 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" Aug 04 16:58:07.217618 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:07.217459 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn" event={"ID":"fa8f63d2-8f98-4711-b570-1ce21a150c73","Type":"ContainerDied","Data":"5d2c808170e9cd6685381cd65ababbfb4a0949d87ccca298f5a680eb19ea2fc9"} Aug 04 16:58:07.217618 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:07.217509 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d2c808170e9cd6685381cd65ababbfb4a0949d87ccca298f5a680eb19ea2fc9" Aug 04 16:58:19.392384 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.392350 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb"] Aug 04 16:58:19.392836 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.392635 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" podUID="f298c5e5-00fa-470d-9942-f9fe63daa204" containerName="payload-processing" containerID="cri-o://5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243" gracePeriod=30 Aug 04 16:58:19.398669 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.398574 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz"] Aug 04 16:58:19.398967 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.398852 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" podUID="2a1266ad-3295-4bae-89b5-735d889f72da" containerName="payload-pre-processing" containerID="cri-o://efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23" gracePeriod=30 Aug 04 16:58:19.539883 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.539842 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn"] Aug 04 16:58:19.542458 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.542417 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bbe0dc-p89pn"] Aug 04 16:58:19.680039 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.680014 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:58:19.706451 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.706418 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:58:19.737978 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.737945 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f298c5e5-00fa-470d-9942-f9fe63daa204-plugins-config-volume\") pod \"f298c5e5-00fa-470d-9942-f9fe63daa204\" (UID: \"f298c5e5-00fa-470d-9942-f9fe63daa204\") " Aug 04 16:58:19.738158 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.738003 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gcc6\" (UniqueName: \"kubernetes.io/projected/f298c5e5-00fa-470d-9942-f9fe63daa204-kube-api-access-6gcc6\") pod \"f298c5e5-00fa-470d-9942-f9fe63daa204\" (UID: \"f298c5e5-00fa-470d-9942-f9fe63daa204\") " Aug 04 16:58:19.738369 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.738344 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f298c5e5-00fa-470d-9942-f9fe63daa204-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f298c5e5-00fa-470d-9942-f9fe63daa204" (UID: "f298c5e5-00fa-470d-9942-f9fe63daa204"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:58:19.738451 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.738432 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f298c5e5-00fa-470d-9942-f9fe63daa204-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:58:19.740084 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.740062 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f298c5e5-00fa-470d-9942-f9fe63daa204-kube-api-access-6gcc6" (OuterVolumeSpecName: "kube-api-access-6gcc6") pod "f298c5e5-00fa-470d-9942-f9fe63daa204" (UID: "f298c5e5-00fa-470d-9942-f9fe63daa204"). InnerVolumeSpecName "kube-api-access-6gcc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:58:19.839074 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.839037 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42kd2\" (UniqueName: \"kubernetes.io/projected/2a1266ad-3295-4bae-89b5-735d889f72da-kube-api-access-42kd2\") pod \"2a1266ad-3295-4bae-89b5-735d889f72da\" (UID: \"2a1266ad-3295-4bae-89b5-735d889f72da\") " Aug 04 16:58:19.839261 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.839145 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1266ad-3295-4bae-89b5-735d889f72da-plugins-config-volume\") pod \"2a1266ad-3295-4bae-89b5-735d889f72da\" (UID: \"2a1266ad-3295-4bae-89b5-735d889f72da\") " Aug 04 16:58:19.839521 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.839496 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a1266ad-3295-4bae-89b5-735d889f72da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a1266ad-3295-4bae-89b5-735d889f72da" (UID: "2a1266ad-3295-4bae-89b5-735d889f72da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:58:19.839582 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.839504 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gcc6\" (UniqueName: \"kubernetes.io/projected/f298c5e5-00fa-470d-9942-f9fe63daa204-kube-api-access-6gcc6\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:58:19.841223 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.841202 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a1266ad-3295-4bae-89b5-735d889f72da-kube-api-access-42kd2" (OuterVolumeSpecName: "kube-api-access-42kd2") pod "2a1266ad-3295-4bae-89b5-735d889f72da" (UID: "2a1266ad-3295-4bae-89b5-735d889f72da"). InnerVolumeSpecName "kube-api-access-42kd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:58:19.940128 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.940033 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1266ad-3295-4bae-89b5-735d889f72da-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:58:19.940128 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:19.940069 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42kd2\" (UniqueName: \"kubernetes.io/projected/2a1266ad-3295-4bae-89b5-735d889f72da-kube-api-access-42kd2\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:58:20.299002 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.298965 2556 generic.go:356] "Generic (PLEG): container finished" podID="f298c5e5-00fa-470d-9942-f9fe63daa204" containerID="5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243" exitCode=0 Aug 04 16:58:20.299190 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.299019 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" Aug 04 16:58:20.299190 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.299048 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" event={"ID":"f298c5e5-00fa-470d-9942-f9fe63daa204","Type":"ContainerDied","Data":"5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243"} Aug 04 16:58:20.299190 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.299117 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb" event={"ID":"f298c5e5-00fa-470d-9942-f9fe63daa204","Type":"ContainerDied","Data":"83c0f12453ef8ddcd5dd6524d6650a6c0329f26834da1a384607b1edc6216b39"} Aug 04 16:58:20.299190 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.299151 2556 scope.go:117] "RemoveContainer" containerID="5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243" Aug 04 16:58:20.300426 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.300401 2556 generic.go:356] "Generic (PLEG): container finished" podID="2a1266ad-3295-4bae-89b5-735d889f72da" containerID="efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23" exitCode=0 Aug 04 16:58:20.300545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.300477 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" event={"ID":"2a1266ad-3295-4bae-89b5-735d889f72da","Type":"ContainerDied","Data":"efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23"} Aug 04 16:58:20.300545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.300520 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" event={"ID":"2a1266ad-3295-4bae-89b5-735d889f72da","Type":"ContainerDied","Data":"8b8a9c78b74b966898f63bbaf11ea7f9ae8b33e0cd2510be7bc57b8813e13ef7"} Aug 04 16:58:20.300545 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.300481 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz" Aug 04 16:58:20.309983 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.309962 2556 scope.go:117] "RemoveContainer" containerID="5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243" Aug 04 16:58:20.310272 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:58:20.310249 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243\": container with ID starting with 5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243 not found: ID does not exist" containerID="5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243" Aug 04 16:58:20.310372 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.310280 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243"} err="failed to get container status \"5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243\": rpc error: code = NotFound desc = could not find container \"5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243\": container with ID starting with 5ad1a63e4c40093c067bac827ad0dea4fdcc4cebb22fc66a4affc4f82dd69243 not found: ID does not exist" Aug 04 16:58:20.310372 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.310297 2556 scope.go:117] "RemoveContainer" containerID="efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23" Aug 04 16:58:20.318574 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.318553 2556 scope.go:117] "RemoveContainer" containerID="efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23" Aug 04 16:58:20.318855 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:58:20.318836 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23\": container with ID starting with efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23 not found: ID does not exist" containerID="efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23" Aug 04 16:58:20.318926 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.318865 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23"} err="failed to get container status \"efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23\": rpc error: code = NotFound desc = could not find container \"efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23\": container with ID starting with efd8d6ba7b1811bbcb87b79d3c0ef4070a64294eeebed2302df275087df5ec23 not found: ID does not exist" Aug 04 16:58:20.325657 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.325631 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb"] Aug 04 16:58:20.329793 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.329771 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bbe0dc-57476bc6c4-j64nb"] Aug 04 16:58:20.344146 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.344123 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz"] Aug 04 16:58:20.349836 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:20.349813 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bbe0dc-595c4d56bd-cswtz"] Aug 04 16:58:21.274094 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:21.274062 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a1266ad-3295-4bae-89b5-735d889f72da" path="/var/lib/kubelet/pods/2a1266ad-3295-4bae-89b5-735d889f72da/volumes" Aug 04 16:58:21.274487 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:21.274417 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f298c5e5-00fa-470d-9942-f9fe63daa204" path="/var/lib/kubelet/pods/f298c5e5-00fa-470d-9942-f9fe63daa204/volumes" Aug 04 16:58:21.274715 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:58:21.274702 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa8f63d2-8f98-4711-b570-1ce21a150c73" path="/var/lib/kubelet/pods/fa8f63d2-8f98-4711-b570-1ce21a150c73/volumes" Aug 04 16:59:18.866719 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:18.866686 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6"] Aug 04 16:59:18.867364 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:18.867034 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" podUID="dcb65243-9127-430b-aefa-8a83de79d5a7" containerName="payload-pre-processing" containerID="cri-o://398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995" gracePeriod=30 Aug 04 16:59:19.137704 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.137679 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:59:19.238835 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.238802 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb65243-9127-430b-aefa-8a83de79d5a7-plugins-config-volume\") pod \"dcb65243-9127-430b-aefa-8a83de79d5a7\" (UID: \"dcb65243-9127-430b-aefa-8a83de79d5a7\") " Aug 04 16:59:19.238997 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.238885 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws282\" (UniqueName: \"kubernetes.io/projected/dcb65243-9127-430b-aefa-8a83de79d5a7-kube-api-access-ws282\") pod \"dcb65243-9127-430b-aefa-8a83de79d5a7\" (UID: \"dcb65243-9127-430b-aefa-8a83de79d5a7\") " Aug 04 16:59:19.239283 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.239253 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcb65243-9127-430b-aefa-8a83de79d5a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dcb65243-9127-430b-aefa-8a83de79d5a7" (UID: "dcb65243-9127-430b-aefa-8a83de79d5a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:59:19.241074 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.241041 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcb65243-9127-430b-aefa-8a83de79d5a7-kube-api-access-ws282" (OuterVolumeSpecName: "kube-api-access-ws282") pod "dcb65243-9127-430b-aefa-8a83de79d5a7" (UID: "dcb65243-9127-430b-aefa-8a83de79d5a7"). InnerVolumeSpecName "kube-api-access-ws282". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:59:19.340289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.340250 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws282\" (UniqueName: \"kubernetes.io/projected/dcb65243-9127-430b-aefa-8a83de79d5a7-kube-api-access-ws282\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:19.340289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.340290 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb65243-9127-430b-aefa-8a83de79d5a7-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:19.638642 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.638610 2556 generic.go:356] "Generic (PLEG): container finished" podID="dcb65243-9127-430b-aefa-8a83de79d5a7" containerID="398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995" exitCode=0 Aug 04 16:59:19.638841 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.638657 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" Aug 04 16:59:19.638841 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.638664 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" event={"ID":"dcb65243-9127-430b-aefa-8a83de79d5a7","Type":"ContainerDied","Data":"398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995"} Aug 04 16:59:19.638841 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.638712 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6" event={"ID":"dcb65243-9127-430b-aefa-8a83de79d5a7","Type":"ContainerDied","Data":"2a6082651f291f1454856719ecfeaa381cde4703c6fcd462df1522fdbea2a481"} Aug 04 16:59:19.638841 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.638760 2556 scope.go:117] "RemoveContainer" containerID="398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995" Aug 04 16:59:19.647431 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.647410 2556 scope.go:117] "RemoveContainer" containerID="398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995" Aug 04 16:59:19.647712 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:59:19.647692 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995\": container with ID starting with 398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995 not found: ID does not exist" containerID="398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995" Aug 04 16:59:19.647809 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.647718 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995"} err="failed to get container status \"398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995\": rpc error: code = NotFound desc = could not find container \"398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995\": container with ID starting with 398d76bad220b9f60ace1ddf4a245073e707ff8a1f1ca79549d1cb0c550d2995 not found: ID does not exist" Aug 04 16:59:19.655888 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.655865 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6"] Aug 04 16:59:19.659292 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:19.659270 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-54f057-5c97c94f76-w4jj6"] Aug 04 16:59:21.273842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:21.273807 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcb65243-9127-430b-aefa-8a83de79d5a7" path="/var/lib/kubelet/pods/dcb65243-9127-430b-aefa-8a83de79d5a7/volumes" Aug 04 16:59:36.331842 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.331803 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28"] Aug 04 16:59:36.332330 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.332075 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" podUID="37e2aa30-834e-4578-a6b3-239e3456f69f" containerName="payload-processing" containerID="cri-o://637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860" gracePeriod=30 Aug 04 16:59:36.338162 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.337879 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8"] Aug 04 16:59:36.338162 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.338122 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" podUID="d910e5c0-7f63-4a32-9956-ab6019773273" containerName="payload-pre-processing" containerID="cri-o://73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895" gracePeriod=30 Aug 04 16:59:36.632919 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.632889 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:59:36.635825 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.635799 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:59:36.716875 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.716838 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d910e5c0-7f63-4a32-9956-ab6019773273-plugins-config-volume\") pod \"d910e5c0-7f63-4a32-9956-ab6019773273\" (UID: \"d910e5c0-7f63-4a32-9956-ab6019773273\") " Aug 04 16:59:36.717046 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.716890 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8hdm\" (UniqueName: \"kubernetes.io/projected/d910e5c0-7f63-4a32-9956-ab6019773273-kube-api-access-c8hdm\") pod \"d910e5c0-7f63-4a32-9956-ab6019773273\" (UID: \"d910e5c0-7f63-4a32-9956-ab6019773273\") " Aug 04 16:59:36.717046 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.716923 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e2aa30-834e-4578-a6b3-239e3456f69f-plugins-config-volume\") pod \"37e2aa30-834e-4578-a6b3-239e3456f69f\" (UID: \"37e2aa30-834e-4578-a6b3-239e3456f69f\") " Aug 04 16:59:36.717046 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.716968 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8mk5\" (UniqueName: \"kubernetes.io/projected/37e2aa30-834e-4578-a6b3-239e3456f69f-kube-api-access-f8mk5\") pod \"37e2aa30-834e-4578-a6b3-239e3456f69f\" (UID: \"37e2aa30-834e-4578-a6b3-239e3456f69f\") " Aug 04 16:59:36.717299 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.717273 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d910e5c0-7f63-4a32-9956-ab6019773273-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d910e5c0-7f63-4a32-9956-ab6019773273" (UID: "d910e5c0-7f63-4a32-9956-ab6019773273"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:59:36.717299 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.717286 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37e2aa30-834e-4578-a6b3-239e3456f69f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "37e2aa30-834e-4578-a6b3-239e3456f69f" (UID: "37e2aa30-834e-4578-a6b3-239e3456f69f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:59:36.719062 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.719037 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37e2aa30-834e-4578-a6b3-239e3456f69f-kube-api-access-f8mk5" (OuterVolumeSpecName: "kube-api-access-f8mk5") pod "37e2aa30-834e-4578-a6b3-239e3456f69f" (UID: "37e2aa30-834e-4578-a6b3-239e3456f69f"). InnerVolumeSpecName "kube-api-access-f8mk5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:59:36.719161 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.719081 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d910e5c0-7f63-4a32-9956-ab6019773273-kube-api-access-c8hdm" (OuterVolumeSpecName: "kube-api-access-c8hdm") pod "d910e5c0-7f63-4a32-9956-ab6019773273" (UID: "d910e5c0-7f63-4a32-9956-ab6019773273"). InnerVolumeSpecName "kube-api-access-c8hdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:59:36.739051 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.739019 2556 generic.go:356] "Generic (PLEG): container finished" podID="d910e5c0-7f63-4a32-9956-ab6019773273" containerID="73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895" exitCode=0 Aug 04 16:59:36.739188 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.739066 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" Aug 04 16:59:36.739188 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.739109 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" event={"ID":"d910e5c0-7f63-4a32-9956-ab6019773273","Type":"ContainerDied","Data":"73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895"} Aug 04 16:59:36.739188 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.739170 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8" event={"ID":"d910e5c0-7f63-4a32-9956-ab6019773273","Type":"ContainerDied","Data":"537f24892e72b77151354f416c5ebc9e8c83ff69c19bfda49a04e6cfc2ca5fff"} Aug 04 16:59:36.739329 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.739198 2556 scope.go:117] "RemoveContainer" containerID="73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895" Aug 04 16:59:36.740366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.740342 2556 generic.go:356] "Generic (PLEG): container finished" podID="37e2aa30-834e-4578-a6b3-239e3456f69f" containerID="637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860" exitCode=0 Aug 04 16:59:36.740439 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.740380 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" Aug 04 16:59:36.740439 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.740392 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" event={"ID":"37e2aa30-834e-4578-a6b3-239e3456f69f","Type":"ContainerDied","Data":"637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860"} Aug 04 16:59:36.740520 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.740446 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28" event={"ID":"37e2aa30-834e-4578-a6b3-239e3456f69f","Type":"ContainerDied","Data":"211d32eb81d992be91bacdcbea44d5d0d4141a9a7faf105f79b811d84ed73de7"} Aug 04 16:59:36.749249 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.749231 2556 scope.go:117] "RemoveContainer" containerID="73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895" Aug 04 16:59:36.749523 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:59:36.749506 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895\": container with ID starting with 73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895 not found: ID does not exist" containerID="73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895" Aug 04 16:59:36.749581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.749532 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895"} err="failed to get container status \"73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895\": rpc error: code = NotFound desc = could not find container \"73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895\": container with ID starting with 73c27995eaa59b6adb9c87aa58a84460330648f981a7629df960bb5839415895 not found: ID does not exist" Aug 04 16:59:36.749581 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.749547 2556 scope.go:117] "RemoveContainer" containerID="637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860" Aug 04 16:59:36.758220 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.758199 2556 scope.go:117] "RemoveContainer" containerID="637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860" Aug 04 16:59:36.758498 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:59:36.758476 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860\": container with ID starting with 637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860 not found: ID does not exist" containerID="637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860" Aug 04 16:59:36.758571 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.758511 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860"} err="failed to get container status \"637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860\": rpc error: code = NotFound desc = could not find container \"637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860\": container with ID starting with 637bee21e86018620bb37a8f5bea334319246742ec369ba523de3e6272a18860 not found: ID does not exist" Aug 04 16:59:36.764508 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.764482 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8"] Aug 04 16:59:36.768366 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.768343 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-90d707-7dcc96fbb4-kg5r8"] Aug 04 16:59:36.778984 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.778958 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28"] Aug 04 16:59:36.782408 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.782383 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-90d707-56b986d878-rnx28"] Aug 04 16:59:36.817957 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.817926 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8hdm\" (UniqueName: \"kubernetes.io/projected/d910e5c0-7f63-4a32-9956-ab6019773273-kube-api-access-c8hdm\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:36.817957 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.817955 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e2aa30-834e-4578-a6b3-239e3456f69f-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:36.818131 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.817966 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8mk5\" (UniqueName: \"kubernetes.io/projected/37e2aa30-834e-4578-a6b3-239e3456f69f-kube-api-access-f8mk5\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:36.818131 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:36.817974 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d910e5c0-7f63-4a32-9956-ab6019773273-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:37.273437 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:37.273401 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37e2aa30-834e-4578-a6b3-239e3456f69f" path="/var/lib/kubelet/pods/37e2aa30-834e-4578-a6b3-239e3456f69f/volumes" Aug 04 16:59:37.273865 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:37.273849 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d910e5c0-7f63-4a32-9956-ab6019773273" path="/var/lib/kubelet/pods/d910e5c0-7f63-4a32-9956-ab6019773273/volumes" Aug 04 16:59:43.282652 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:43.282616 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5"] Aug 04 16:59:43.283142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:43.282897 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" containerID="cri-o://8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430" gracePeriod=30 Aug 04 16:59:46.242289 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:46.242244 2556 patch_prober.go:28] interesting pod/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 16:59:46.242666 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:46.242318 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:59:48.540927 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.540899 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:59:48.632142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632052 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acfbe88c-858a-47f4-a927-c69fe0fedb55-istiod-ca-cert\") pod \"acfbe88c-858a-47f4-a927-c69fe0fedb55\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " Aug 04 16:59:48.632142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632088 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-token\") pod \"acfbe88c-858a-47f4-a927-c69fe0fedb55\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " Aug 04 16:59:48.632142 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632131 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr5jw\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-kube-api-access-fr5jw\") pod \"acfbe88c-858a-47f4-a927-c69fe0fedb55\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " Aug 04 16:59:48.632410 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632242 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-certs\") pod \"acfbe88c-858a-47f4-a927-c69fe0fedb55\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " Aug 04 16:59:48.632410 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632279 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-data\") pod \"acfbe88c-858a-47f4-a927-c69fe0fedb55\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " Aug 04 16:59:48.632410 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632300 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-credential-socket\") pod \"acfbe88c-858a-47f4-a927-c69fe0fedb55\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " Aug 04 16:59:48.632410 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632392 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-socket\") pod \"acfbe88c-858a-47f4-a927-c69fe0fedb55\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " Aug 04 16:59:48.632610 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632419 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-envoy\") pod \"acfbe88c-858a-47f4-a927-c69fe0fedb55\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " Aug 04 16:59:48.632610 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632447 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-podinfo\") pod \"acfbe88c-858a-47f4-a927-c69fe0fedb55\" (UID: \"acfbe88c-858a-47f4-a927-c69fe0fedb55\") " Aug 04 16:59:48.632610 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632550 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "acfbe88c-858a-47f4-a927-c69fe0fedb55" (UID: "acfbe88c-858a-47f4-a927-c69fe0fedb55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:59:48.632808 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632629 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acfbe88c-858a-47f4-a927-c69fe0fedb55-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "acfbe88c-858a-47f4-a927-c69fe0fedb55" (UID: "acfbe88c-858a-47f4-a927-c69fe0fedb55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:59:48.632808 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632657 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-data" (OuterVolumeSpecName: "istio-data") pod "acfbe88c-858a-47f4-a927-c69fe0fedb55" (UID: "acfbe88c-858a-47f4-a927-c69fe0fedb55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:59:48.632808 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632763 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "acfbe88c-858a-47f4-a927-c69fe0fedb55" (UID: "acfbe88c-858a-47f4-a927-c69fe0fedb55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:59:48.632979 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632813 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "acfbe88c-858a-47f4-a927-c69fe0fedb55" (UID: "acfbe88c-858a-47f4-a927-c69fe0fedb55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:59:48.632979 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632941 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:48.632979 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632955 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acfbe88c-858a-47f4-a927-c69fe0fedb55-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:48.632979 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632964 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:48.632979 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632972 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:48.632979 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.632980 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:48.634864 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.634841 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "acfbe88c-858a-47f4-a927-c69fe0fedb55" (UID: "acfbe88c-858a-47f4-a927-c69fe0fedb55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:59:48.634998 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.634893 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "acfbe88c-858a-47f4-a927-c69fe0fedb55" (UID: "acfbe88c-858a-47f4-a927-c69fe0fedb55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:59:48.634998 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.634960 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-token" (OuterVolumeSpecName: "istio-token") pod "acfbe88c-858a-47f4-a927-c69fe0fedb55" (UID: "acfbe88c-858a-47f4-a927-c69fe0fedb55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:59:48.635084 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.635059 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-kube-api-access-fr5jw" (OuterVolumeSpecName: "kube-api-access-fr5jw") pod "acfbe88c-858a-47f4-a927-c69fe0fedb55" (UID: "acfbe88c-858a-47f4-a927-c69fe0fedb55"). InnerVolumeSpecName "kube-api-access-fr5jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:59:48.733994 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.733956 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:48.733994 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.733985 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr5jw\" (UniqueName: \"kubernetes.io/projected/acfbe88c-858a-47f4-a927-c69fe0fedb55-kube-api-access-fr5jw\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:48.733994 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.733997 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:48.734240 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.734008 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acfbe88c-858a-47f4-a927-c69fe0fedb55-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:48.809310 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.809278 2556 generic.go:356] "Generic (PLEG): container finished" podID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerID="8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430" exitCode=0 Aug 04 16:59:48.809488 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.809324 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" Aug 04 16:59:48.809488 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.809366 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" event={"ID":"acfbe88c-858a-47f4-a927-c69fe0fedb55","Type":"ContainerDied","Data":"8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430"} Aug 04 16:59:48.809488 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.809421 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5" event={"ID":"acfbe88c-858a-47f4-a927-c69fe0fedb55","Type":"ContainerDied","Data":"67930c8315c082a8e999b8f73095432519c0ea45d665074af5c6e7711bde79ee"} Aug 04 16:59:48.809488 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.809445 2556 scope.go:117] "RemoveContainer" containerID="8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430" Aug 04 16:59:48.818673 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.818652 2556 scope.go:117] "RemoveContainer" containerID="52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461" Aug 04 16:59:48.826812 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.826793 2556 scope.go:117] "RemoveContainer" containerID="8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430" Aug 04 16:59:48.827066 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:59:48.827049 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430\": container with ID starting with 8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430 not found: ID does not exist" containerID="8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430" Aug 04 16:59:48.827122 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.827073 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430"} err="failed to get container status \"8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430\": rpc error: code = NotFound desc = could not find container \"8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430\": container with ID starting with 8afcbf7d1b5890340df49a39d1141619ae809b8faec2f8f69c7e0d6d3824b430 not found: ID does not exist" Aug 04 16:59:48.827122 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.827089 2556 scope.go:117] "RemoveContainer" containerID="52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461" Aug 04 16:59:48.827291 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:59:48.827277 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461\": container with ID starting with 52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461 not found: ID does not exist" containerID="52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461" Aug 04 16:59:48.827333 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.827294 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461"} err="failed to get container status \"52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461\": rpc error: code = NotFound desc = could not find container \"52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461\": container with ID starting with 52bc531289b05a7726dd733d3fe539ce0eeba97db38b320d8d2703b99155c461 not found: ID does not exist" Aug 04 16:59:48.833987 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.833961 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5"] Aug 04 16:59:48.838393 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.838367 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-90d707-openshift-default-7b88c68f5b-cr9q5"] Aug 04 16:59:48.957065 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.956980 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr"] Aug 04 16:59:48.957261 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.957215 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" podUID="89428bbc-eb8a-45bf-9299-35e446d987ed" containerName="payload-processing" containerID="cri-o://ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4" gracePeriod=30 Aug 04 16:59:48.964297 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.963778 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9"] Aug 04 16:59:48.964297 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:48.964014 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" podUID="f7c8d539-1e12-436f-94ea-e7a7b2a37f73" containerName="payload-pre-processing" containerID="cri-o://bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1" gracePeriod=30 Aug 04 16:59:49.246821 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.246792 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:59:49.262555 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.262527 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:59:49.276528 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.276499 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" path="/var/lib/kubelet/pods/acfbe88c-858a-47f4-a927-c69fe0fedb55/volumes" Aug 04 16:59:49.339954 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.339921 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgvsh\" (UniqueName: \"kubernetes.io/projected/89428bbc-eb8a-45bf-9299-35e446d987ed-kube-api-access-pgvsh\") pod \"89428bbc-eb8a-45bf-9299-35e446d987ed\" (UID: \"89428bbc-eb8a-45bf-9299-35e446d987ed\") " Aug 04 16:59:49.340094 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.339999 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-plugins-config-volume\") pod \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\" (UID: \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\") " Aug 04 16:59:49.340094 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.340030 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq7tf\" (UniqueName: \"kubernetes.io/projected/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-kube-api-access-pq7tf\") pod \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\" (UID: \"f7c8d539-1e12-436f-94ea-e7a7b2a37f73\") " Aug 04 16:59:49.340094 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.340057 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89428bbc-eb8a-45bf-9299-35e446d987ed-plugins-config-volume\") pod \"89428bbc-eb8a-45bf-9299-35e446d987ed\" (UID: \"89428bbc-eb8a-45bf-9299-35e446d987ed\") " Aug 04 16:59:49.340431 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.340404 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7c8d539-1e12-436f-94ea-e7a7b2a37f73" (UID: "f7c8d539-1e12-436f-94ea-e7a7b2a37f73"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:59:49.340568 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.340490 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89428bbc-eb8a-45bf-9299-35e446d987ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89428bbc-eb8a-45bf-9299-35e446d987ed" (UID: "89428bbc-eb8a-45bf-9299-35e446d987ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:59:49.341860 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.341833 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89428bbc-eb8a-45bf-9299-35e446d987ed-kube-api-access-pgvsh" (OuterVolumeSpecName: "kube-api-access-pgvsh") pod "89428bbc-eb8a-45bf-9299-35e446d987ed" (UID: "89428bbc-eb8a-45bf-9299-35e446d987ed"). InnerVolumeSpecName "kube-api-access-pgvsh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:59:49.342013 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.341996 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-kube-api-access-pq7tf" (OuterVolumeSpecName: "kube-api-access-pq7tf") pod "f7c8d539-1e12-436f-94ea-e7a7b2a37f73" (UID: "f7c8d539-1e12-436f-94ea-e7a7b2a37f73"). InnerVolumeSpecName "kube-api-access-pq7tf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:59:49.441155 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.441121 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgvsh\" (UniqueName: \"kubernetes.io/projected/89428bbc-eb8a-45bf-9299-35e446d987ed-kube-api-access-pgvsh\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:49.441155 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.441150 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:49.441155 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.441161 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq7tf\" (UniqueName: \"kubernetes.io/projected/f7c8d539-1e12-436f-94ea-e7a7b2a37f73-kube-api-access-pq7tf\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:49.441393 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.441171 2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89428bbc-eb8a-45bf-9299-35e446d987ed-plugins-config-volume\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 16:59:49.816315 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.816277 2556 generic.go:356] "Generic (PLEG): container finished" podID="f7c8d539-1e12-436f-94ea-e7a7b2a37f73" containerID="bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1" exitCode=0 Aug 04 16:59:49.816772 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.816320 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" Aug 04 16:59:49.816772 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.816363 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" event={"ID":"f7c8d539-1e12-436f-94ea-e7a7b2a37f73","Type":"ContainerDied","Data":"bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1"} Aug 04 16:59:49.816772 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.816420 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9" event={"ID":"f7c8d539-1e12-436f-94ea-e7a7b2a37f73","Type":"ContainerDied","Data":"6e5206abdf419387e2bfe8b31e654dac00e8ec90cb1eea89b2ab9d858e73065a"} Aug 04 16:59:49.816772 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.816448 2556 scope.go:117] "RemoveContainer" containerID="bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1" Aug 04 16:59:49.818799 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.818773 2556 generic.go:356] "Generic (PLEG): container finished" podID="89428bbc-eb8a-45bf-9299-35e446d987ed" containerID="ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4" exitCode=0 Aug 04 16:59:49.818901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.818818 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" event={"ID":"89428bbc-eb8a-45bf-9299-35e446d987ed","Type":"ContainerDied","Data":"ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4"} Aug 04 16:59:49.818901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.818859 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" event={"ID":"89428bbc-eb8a-45bf-9299-35e446d987ed","Type":"ContainerDied","Data":"326548c1315bc55245c83cd74cf7059ad819543f8104ca8c038a648951cd013a"} Aug 04 16:59:49.818901 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.818823 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr" Aug 04 16:59:49.825972 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.825949 2556 scope.go:117] "RemoveContainer" containerID="bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1" Aug 04 16:59:49.826266 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:59:49.826240 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1\": container with ID starting with bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1 not found: ID does not exist" containerID="bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1" Aug 04 16:59:49.826344 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.826274 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1"} err="failed to get container status \"bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1\": rpc error: code = NotFound desc = could not find container \"bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1\": container with ID starting with bb4093fef4b83cad3269434714bd401f57d1a5cafd0bf77eb5be3ec3bd39c2f1 not found: ID does not exist" Aug 04 16:59:49.826344 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.826296 2556 scope.go:117] "RemoveContainer" containerID="ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4" Aug 04 16:59:49.835229 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.835212 2556 scope.go:117] "RemoveContainer" containerID="ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4" Aug 04 16:59:49.835473 ip-10-0-132-227 kubenswrapper[2556]: E0804 16:59:49.835455 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4\": container with ID starting with ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4 not found: ID does not exist" containerID="ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4" Aug 04 16:59:49.835527 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.835481 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4"} err="failed to get container status \"ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4\": rpc error: code = NotFound desc = could not find container \"ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4\": container with ID starting with ba7fb38b7c5120ff706ebebe5f4cd1d8398c054cf2e8051dd62d4f1f3e0375e4 not found: ID does not exist" Aug 04 16:59:49.843767 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.843740 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr"] Aug 04 16:59:49.850659 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.850634 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5a3cb8-8769dc7c8-bm5vr"] Aug 04 16:59:49.869272 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.869243 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9"] Aug 04 16:59:49.873300 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:49.873272 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5a3cb8-6b8f49cf4d-xthp9"] Aug 04 16:59:51.274227 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:51.274192 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89428bbc-eb8a-45bf-9299-35e446d987ed" path="/var/lib/kubelet/pods/89428bbc-eb8a-45bf-9299-35e446d987ed/volumes" Aug 04 16:59:51.274694 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:51.274553 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7c8d539-1e12-436f-94ea-e7a7b2a37f73" path="/var/lib/kubelet/pods/f7c8d539-1e12-436f-94ea-e7a7b2a37f73/volumes" Aug 04 16:59:55.941547 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:55.941511 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7"] Aug 04 16:59:55.942048 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:55.941807 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" containerID="cri-o://9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a" gracePeriod=30 Aug 04 16:59:59.987537 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:59.987495 2556 patch_prober.go:28] interesting pod/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 16:59:59.987955 ip-10-0-132-227 kubenswrapper[2556]: I0804 16:59:59.987571 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 17:00:01.205177 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.205151 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 17:00:01.254259 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254229 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-token\") pod \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " Aug 04 17:00:01.254443 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254268 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-certs\") pod \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " Aug 04 17:00:01.254443 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254298 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-credential-socket\") pod \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " Aug 04 17:00:01.254443 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254376 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-data\") pod \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " Aug 04 17:00:01.254443 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254402 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-envoy\") pod \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " Aug 04 17:00:01.254443 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254440 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-socket\") pod \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " Aug 04 17:00:01.254738 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254474 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istiod-ca-cert\") pod \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " Aug 04 17:00:01.254738 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254513 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf925\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-kube-api-access-rf925\") pod \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " Aug 04 17:00:01.254738 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254557 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" (UID: "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 17:00:01.254738 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254564 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-podinfo\") pod \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\" (UID: \"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b\") " Aug 04 17:00:01.254738 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254656 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" (UID: "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 17:00:01.254983 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254792 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" (UID: "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 17:00:01.254983 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.254848 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" (UID: "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 17:00:01.255062 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.255015 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-data\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 17:00:01.255062 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.255035 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 17:00:01.255062 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.255051 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-workload-certs\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 17:00:01.255202 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.255065 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-credential-socket\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 17:00:01.255599 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.255568 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" (UID: "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 17:00:01.256784 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.256752 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" (UID: "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 17:00:01.256903 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.256806 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" (UID: "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 17:00:01.256903 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.256855 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-kube-api-access-rf925" (OuterVolumeSpecName: "kube-api-access-rf925") pod "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" (UID: "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b"). InnerVolumeSpecName "kube-api-access-rf925". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 17:00:01.257223 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.257208 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" (UID: "a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 17:00:01.355766 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.355708 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-token\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 17:00:01.355766 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.355767 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-envoy\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 17:00:01.355977 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.355780 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istiod-ca-cert\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 17:00:01.355977 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.355793 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf925\" (UniqueName: \"kubernetes.io/projected/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-kube-api-access-rf925\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 17:00:01.355977 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.355805 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b-istio-podinfo\") on node \"ip-10-0-132-227.ec2.internal\" DevicePath \"\"" Aug 04 17:00:01.891152 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.891115 2556 generic.go:356] "Generic (PLEG): container finished" podID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerID="9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a" exitCode=0 Aug 04 17:00:01.891331 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.891166 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" Aug 04 17:00:01.891331 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.891191 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" event={"ID":"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b","Type":"ContainerDied","Data":"9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a"} Aug 04 17:00:01.891331 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.891245 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7" event={"ID":"a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b","Type":"ContainerDied","Data":"3c28fb273de3f0a6c05234417d803c0212791ec5c03d63339d4c16cde3aee0d1"} Aug 04 17:00:01.891331 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.891268 2556 scope.go:117] "RemoveContainer" containerID="9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a" Aug 04 17:00:01.900182 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.900163 2556 scope.go:117] "RemoveContainer" containerID="0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2" Aug 04 17:00:01.909081 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.909059 2556 scope.go:117] "RemoveContainer" containerID="9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a" Aug 04 17:00:01.909411 ip-10-0-132-227 kubenswrapper[2556]: E0804 17:00:01.909390 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a\": container with ID starting with 9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a not found: ID does not exist" containerID="9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a" Aug 04 17:00:01.909473 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.909422 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a"} err="failed to get container status \"9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a\": rpc error: code = NotFound desc = could not find container \"9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a\": container with ID starting with 9ac2e006fddf67e22016c20c10ddbe2451966e813b727c445f5d448404ae898a not found: ID does not exist" Aug 04 17:00:01.909473 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.909442 2556 scope.go:117] "RemoveContainer" containerID="0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2" Aug 04 17:00:01.909698 ip-10-0-132-227 kubenswrapper[2556]: E0804 17:00:01.909682 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2\": container with ID starting with 0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2 not found: ID does not exist" containerID="0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2" Aug 04 17:00:01.909769 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.909702 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2"} err="failed to get container status \"0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2\": rpc error: code = NotFound desc = could not find container \"0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2\": container with ID starting with 0e597ffe176579a255db64a572de0e54c95f26b986fc03e131a5137a5925a9e2 not found: ID does not exist" Aug 04 17:00:01.910212 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.910194 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7"] Aug 04 17:00:01.915180 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:01.915157 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-5a3cb8-openshift-default-7c7c9b54d8-bcbx7"] Aug 04 17:00:02.373353 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:02.373321 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5z45r_18303a78-30fe-4a3d-bbc6-d30bc4604c51/manager/0.log" Aug 04 17:00:02.616249 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:02.616169 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d474f94c9-frkfm_d4792e00-f3e3-4ddf-9bf6-6e39924d9129/manager/0.log" Aug 04 17:00:03.105701 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:03.105674 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfb598967-fv7cz_114027f0-89cf-4672-9f5e-44d5dab1e346/copy-manifests/0.log" Aug 04 17:00:03.113346 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:03.113316 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfb598967-fv7cz_114027f0-89cf-4672-9f5e-44d5dab1e346/manager/0.log" Aug 04 17:00:03.273163 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:03.273128 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" path="/var/lib/kubelet/pods/a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b/volumes" Aug 04 17:00:03.514128 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:03.514088 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd"] Aug 04 17:00:03.515836 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:03.515816 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764335-6b4gd"] Aug 04 17:00:03.861680 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:03.861608 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764350-g4g87_12d587de-1a08-4921-8635-b4a4c02cd963/cleanup/0.log" Aug 04 17:00:04.232968 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:04.232892 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-rbzc8_32de1bf2-b2b9-4eb8-8ca5-db7fc4c01d67/postgres/0.log" Aug 04 17:00:05.273227 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:05.273192 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e663aa28-702d-4637-b668-12821de6232f" path="/var/lib/kubelet/pods/e663aa28-702d-4637-b668-12821de6232f/volumes" Aug 04 17:00:05.319554 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:05.319522 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-ml5nm_7c736b8e-2fec-4380-ad41-e9c5ddd38939/manager/0.log" Aug 04 17:00:05.796194 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:05.796162 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-68l6x_ea4f1663-cdd9-4299-8c6b-19917b0cb9aa/manager/0.log" Aug 04 17:00:06.170137 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:06.170059 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k_8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227/istio-proxy/0.log" Aug 04 17:00:06.648921 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:06.648889 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7gqdr_9f3fb966-8f4b-4709-bbbf-1f45529c4eaf/istio-proxy/1.log" Aug 04 17:00:06.884471 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:06.884438 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6vftr_494ec520-130e-4ca2-a9ca-4f11a19f69eb/payload-processing/0.log" Aug 04 17:00:07.352459 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:07.352428 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-cv48v_8857eb2c-576d-42d7-87ec-9bfdaa387011/main/0.log" Aug 04 17:00:07.466025 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:07.465997 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-48cgv_6ab4cf74-591e-412d-9416-e9dcee87f394/main/0.log" Aug 04 17:00:11.881922 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:11.881889 2556 scope.go:117] "RemoveContainer" containerID="e0a5b5124a8c03303c7c9304e2875e02608e5e2aa0da06c1d92bb3ffb37c4fe4" Aug 04 17:00:14.041201 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041161 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b2bcd/must-gather-tw8hj"] Aug 04 17:00:14.041578 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041544 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" Aug 04 17:00:14.041578 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041554 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" Aug 04 17:00:14.041578 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041562 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c8d539-1e12-436f-94ea-e7a7b2a37f73" containerName="payload-pre-processing" Aug 04 17:00:14.041578 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041569 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c8d539-1e12-436f-94ea-e7a7b2a37f73" containerName="payload-pre-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041579 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a1266ad-3295-4bae-89b5-735d889f72da" containerName="payload-pre-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041586 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a1266ad-3295-4bae-89b5-735d889f72da" containerName="payload-pre-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041594 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d910e5c0-7f63-4a32-9956-ab6019773273" containerName="payload-pre-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041600 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d910e5c0-7f63-4a32-9956-ab6019773273" containerName="payload-pre-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041607 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041613 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041628 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89428bbc-eb8a-45bf-9299-35e446d987ed" containerName="payload-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041633 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="89428bbc-eb8a-45bf-9299-35e446d987ed" containerName="payload-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041641 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcb65243-9127-430b-aefa-8a83de79d5a7" containerName="payload-pre-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041645 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcb65243-9127-430b-aefa-8a83de79d5a7" containerName="payload-pre-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041657 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37e2aa30-834e-4578-a6b3-239e3456f69f" containerName="payload-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041662 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="37e2aa30-834e-4578-a6b3-239e3456f69f" containerName="payload-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041668 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f298c5e5-00fa-470d-9942-f9fe63daa204" containerName="payload-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041673 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f298c5e5-00fa-470d-9942-f9fe63daa204" containerName="payload-processing" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041678 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa8f63d2-8f98-4711-b570-1ce21a150c73" containerName="revoke-keys" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041683 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa8f63d2-8f98-4711-b570-1ce21a150c73" containerName="revoke-keys" Aug 04 17:00:14.041712 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041751 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041759 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcb65243-9127-430b-aefa-8a83de79d5a7" containerName="payload-pre-processing" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041767 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="37e2aa30-834e-4578-a6b3-239e3456f69f" containerName="payload-processing" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041773 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041778 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041784 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041789 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c8d539-1e12-436f-94ea-e7a7b2a37f73" containerName="payload-pre-processing" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041795 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d910e5c0-7f63-4a32-9956-ab6019773273" containerName="payload-pre-processing" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041800 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f298c5e5-00fa-470d-9942-f9fe63daa204" containerName="payload-processing" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041805 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a1266ad-3295-4bae-89b5-735d889f72da" containerName="payload-pre-processing" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041812 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="89428bbc-eb8a-45bf-9299-35e446d987ed" containerName="payload-processing" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041819 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa8f63d2-8f98-4711-b570-1ce21a150c73" containerName="revoke-keys" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041881 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041886 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="acfbe88c-858a-47f4-a927-c69fe0fedb55" containerName="istio-proxy" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041896 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" Aug 04 17:00:14.042261 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.041901 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f78e2e-bdfe-44ad-83ae-4e7ac5a27c1b" containerName="istio-proxy" Aug 04 17:00:14.047279 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.047257 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b2bcd/must-gather-tw8hj" Aug 04 17:00:14.049970 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.049939 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b2bcd\"/\"kube-root-ca.crt\"" Aug 04 17:00:14.050986 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.050963 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b2bcd\"/\"default-dockercfg-5lghf\"" Aug 04 17:00:14.051086 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.050966 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b2bcd\"/\"openshift-service-ca.crt\"" Aug 04 17:00:14.057271 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.057249 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b2bcd/must-gather-tw8hj"] Aug 04 17:00:14.170446 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.170408 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/01743515-7f85-4976-9ae8-3a555412acc0-must-gather-output\") pod \"must-gather-tw8hj\" (UID: \"01743515-7f85-4976-9ae8-3a555412acc0\") " pod="openshift-must-gather-b2bcd/must-gather-tw8hj" Aug 04 17:00:14.170617 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.170527 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2864\" (UniqueName: \"kubernetes.io/projected/01743515-7f85-4976-9ae8-3a555412acc0-kube-api-access-p2864\") pod \"must-gather-tw8hj\" (UID: \"01743515-7f85-4976-9ae8-3a555412acc0\") " pod="openshift-must-gather-b2bcd/must-gather-tw8hj" Aug 04 17:00:14.271067 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.271027 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/01743515-7f85-4976-9ae8-3a555412acc0-must-gather-output\") pod \"must-gather-tw8hj\" (UID: \"01743515-7f85-4976-9ae8-3a555412acc0\") " pod="openshift-must-gather-b2bcd/must-gather-tw8hj" Aug 04 17:00:14.271234 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.271108 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2864\" (UniqueName: \"kubernetes.io/projected/01743515-7f85-4976-9ae8-3a555412acc0-kube-api-access-p2864\") pod \"must-gather-tw8hj\" (UID: \"01743515-7f85-4976-9ae8-3a555412acc0\") " pod="openshift-must-gather-b2bcd/must-gather-tw8hj" Aug 04 17:00:14.271336 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.271316 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/01743515-7f85-4976-9ae8-3a555412acc0-must-gather-output\") pod \"must-gather-tw8hj\" (UID: \"01743515-7f85-4976-9ae8-3a555412acc0\") " pod="openshift-must-gather-b2bcd/must-gather-tw8hj" Aug 04 17:00:14.279696 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.279667 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2864\" (UniqueName: \"kubernetes.io/projected/01743515-7f85-4976-9ae8-3a555412acc0-kube-api-access-p2864\") pod \"must-gather-tw8hj\" (UID: \"01743515-7f85-4976-9ae8-3a555412acc0\") " pod="openshift-must-gather-b2bcd/must-gather-tw8hj" Aug 04 17:00:14.357521 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.357432 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b2bcd/must-gather-tw8hj" Aug 04 17:00:14.496939 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.496912 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b2bcd/must-gather-tw8hj"] Aug 04 17:00:14.498501 ip-10-0-132-227 kubenswrapper[2556]: W0804 17:00:14.498470 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01743515_7f85_4976_9ae8_3a555412acc0.slice/crio-dcc7cfca4b801258d4f5d4ebd57b9c248af394e5c61408ccd498f3b37a0693bd WatchSource:0}: Error finding container dcc7cfca4b801258d4f5d4ebd57b9c248af394e5c61408ccd498f3b37a0693bd: Status 404 returned error can't find the container with id dcc7cfca4b801258d4f5d4ebd57b9c248af394e5c61408ccd498f3b37a0693bd Aug 04 17:00:14.500257 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.500239 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 17:00:14.964566 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:14.964522 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b2bcd/must-gather-tw8hj" event={"ID":"01743515-7f85-4976-9ae8-3a555412acc0","Type":"ContainerStarted","Data":"dcc7cfca4b801258d4f5d4ebd57b9c248af394e5c61408ccd498f3b37a0693bd"} Aug 04 17:00:15.974020 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:15.973977 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b2bcd/must-gather-tw8hj" event={"ID":"01743515-7f85-4976-9ae8-3a555412acc0","Type":"ContainerStarted","Data":"7afefb8a0bc11f3979dd2f388a7430a36044403be56437e28c21018c93f4932d"} Aug 04 17:00:15.974470 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:15.974043 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b2bcd/must-gather-tw8hj" event={"ID":"01743515-7f85-4976-9ae8-3a555412acc0","Type":"ContainerStarted","Data":"382d37f5f5640a00146cd47713d4ad0580215c834085ae371867ac5526730c8d"} Aug 04 17:00:15.991950 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:15.989481 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b2bcd/must-gather-tw8hj" podStartSLOduration=1.253347244 podStartE2EDuration="1.989459615s" totalImagesPullingTime="736.112371ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:00:15.989081751 +0000 UTC m=+3845.311505232" watchObservedRunningTime="2026-08-04 17:00:15.989459615 +0000 UTC m=+3845.311883096" Aug 04 17:00:16.841276 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:16.841244 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-986t7_b9f5fe66-9376-47a2-b714-de84ffb90ad8/global-pull-secret-syncer/0.log" Aug 04 17:00:17.050059 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:17.050013 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nrkz8_7ad081f0-d1a9-4890-b1e9-4784a2f71d2a/konnectivity-agent/0.log" Aug 04 17:00:17.074280 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:17.074251 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-227.ec2.internal_ef5790e9892603695c64fc93d2f5fff7/haproxy/0.log" Aug 04 17:00:22.927951 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:22.927871 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-ml5nm_7c736b8e-2fec-4380-ad41-e9c5ddd38939/manager/0.log" Aug 04 17:00:23.180625 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:23.180541 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-68l6x_ea4f1663-cdd9-4299-8c6b-19917b0cb9aa/manager/0.log" Aug 04 17:00:24.876286 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:24.876217 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-kgblb_259aed9c-29fa-4a34-a9e6-8c1466819bb2/kube-state-metrics/0.log" Aug 04 17:00:24.893100 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:24.893064 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-kgblb_259aed9c-29fa-4a34-a9e6-8c1466819bb2/kube-rbac-proxy-main/0.log" Aug 04 17:00:24.914350 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:24.914320 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-kgblb_259aed9c-29fa-4a34-a9e6-8c1466819bb2/kube-rbac-proxy-self/0.log" Aug 04 17:00:24.940113 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:24.940075 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-69cb84cf77-sq8nt_3a5c2c18-b164-4453-8d06-5759538038cd/metrics-server/0.log" Aug 04 17:00:25.154472 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.154440 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ztb4x_3443e0ca-fb65-47ef-86ba-c3bc535fac61/node-exporter/0.log" Aug 04 17:00:25.185550 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.185522 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ztb4x_3443e0ca-fb65-47ef-86ba-c3bc535fac61/kube-rbac-proxy/0.log" Aug 04 17:00:25.212194 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.212166 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ztb4x_3443e0ca-fb65-47ef-86ba-c3bc535fac61/init-textfile/0.log" Aug 04 17:00:25.336941 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.336908 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c5eb557-35d6-4de5-9111-ca54d47cd01f/prometheus/0.log" Aug 04 17:00:25.359842 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.359811 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c5eb557-35d6-4de5-9111-ca54d47cd01f/config-reloader/0.log" Aug 04 17:00:25.383287 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.383256 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c5eb557-35d6-4de5-9111-ca54d47cd01f/thanos-sidecar/0.log" Aug 04 17:00:25.405820 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.405736 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c5eb557-35d6-4de5-9111-ca54d47cd01f/kube-rbac-proxy-web/0.log" Aug 04 17:00:25.440590 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.440554 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c5eb557-35d6-4de5-9111-ca54d47cd01f/kube-rbac-proxy/0.log" Aug 04 17:00:25.463742 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.463702 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c5eb557-35d6-4de5-9111-ca54d47cd01f/kube-rbac-proxy-thanos/0.log" Aug 04 17:00:25.487023 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.486997 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c5eb557-35d6-4de5-9111-ca54d47cd01f/init-config-reloader/0.log" Aug 04 17:00:25.514854 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.514821 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-lrrmg_fe0db590-0b17-4aaa-b9da-df4837a4a0f0/prometheus-operator/0.log" Aug 04 17:00:25.534264 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.534231 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-lrrmg_fe0db590-0b17-4aaa-b9da-df4837a4a0f0/kube-rbac-proxy/0.log" Aug 04 17:00:25.673800 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.673676 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6967765754-rbtjx_527c1a02-0e18-423b-9d09-f85c4b4dd941/thanos-query/0.log" Aug 04 17:00:25.695081 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.695043 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6967765754-rbtjx_527c1a02-0e18-423b-9d09-f85c4b4dd941/kube-rbac-proxy-web/0.log" Aug 04 17:00:25.716115 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.716030 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6967765754-rbtjx_527c1a02-0e18-423b-9d09-f85c4b4dd941/kube-rbac-proxy/0.log" Aug 04 17:00:25.739470 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.739442 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6967765754-rbtjx_527c1a02-0e18-423b-9d09-f85c4b4dd941/prom-label-proxy/0.log" Aug 04 17:00:25.760118 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.760084 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6967765754-rbtjx_527c1a02-0e18-423b-9d09-f85c4b4dd941/kube-rbac-proxy-rules/0.log" Aug 04 17:00:25.784079 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:25.784051 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6967765754-rbtjx_527c1a02-0e18-423b-9d09-f85c4b4dd941/kube-rbac-proxy-metrics/0.log" Aug 04 17:00:26.949254 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:26.949214 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-mgnfq_76f97e54-1257-4ba8-9033-3ab3b44b9eb8/networking-console-plugin/0.log" Aug 04 17:00:28.080747 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.080690 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-h82zl_0e52402b-8a15-412e-87a2-598b4984aae8/download-server/0.log" Aug 04 17:00:28.125504 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.125463 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm"] Aug 04 17:00:28.131469 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.131442 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.139077 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.139046 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm"] Aug 04 17:00:28.222530 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.222478 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-podres\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.222867 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.222844 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-sys\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.223090 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.223070 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-lib-modules\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.223318 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.223300 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5rkm\" (UniqueName: \"kubernetes.io/projected/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-kube-api-access-l5rkm\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.223508 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.223474 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-proc\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.325054 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.325018 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-proc\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.325269 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.325084 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-podres\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.325269 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.325110 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-sys\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.325269 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.325171 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-lib-modules\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.325269 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.325183 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-proc\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.325269 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.325230 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-podres\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.325530 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.325305 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5rkm\" (UniqueName: \"kubernetes.io/projected/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-kube-api-access-l5rkm\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.325530 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.325318 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-lib-modules\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.325530 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.325366 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-sys\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.333865 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.333781 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5rkm\" (UniqueName: \"kubernetes.io/projected/c561b940-05f3-4c58-ba97-5e5bd7c1c7c2-kube-api-access-l5rkm\") pod \"perf-node-gather-daemonset-jhvpm\" (UID: \"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2\") " pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.451965 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.451926 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:28.663626 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:28.663507 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm"] Aug 04 17:00:28.665926 ip-10-0-132-227 kubenswrapper[2556]: W0804 17:00:28.665887 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc561b940_05f3_4c58_ba97_5e5bd7c1c7c2.slice/crio-6f9b0f9f962201be7a26c2206186656320d9f4265730b261f9ae2ba810aba612 WatchSource:0}: Error finding container 6f9b0f9f962201be7a26c2206186656320d9f4265730b261f9ae2ba810aba612: Status 404 returned error can't find the container with id 6f9b0f9f962201be7a26c2206186656320d9f4265730b261f9ae2ba810aba612 Aug 04 17:00:29.081600 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:29.081562 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" event={"ID":"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2","Type":"ContainerStarted","Data":"16a06b85b2240a86a0f31f56592a70d5fbc56353759651d265a0b437d8b0b544"} Aug 04 17:00:29.082002 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:29.081650 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:29.082002 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:29.081671 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" event={"ID":"c561b940-05f3-4c58-ba97-5e5bd7c1c7c2","Type":"ContainerStarted","Data":"6f9b0f9f962201be7a26c2206186656320d9f4265730b261f9ae2ba810aba612"} Aug 04 17:00:29.098135 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:29.098040 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" podStartSLOduration=1.098020452 podStartE2EDuration="1.098020452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:00:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:00:29.096798736 +0000 UTC m=+3858.419222207" watchObservedRunningTime="2026-08-04 17:00:29.098020452 +0000 UTC m=+3858.420443933" Aug 04 17:00:29.843032 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:29.843001 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ft67f_26589d8e-4d36-466c-993c-2d0fb23c2817/dns/0.log" Aug 04 17:00:29.862798 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:29.862767 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ft67f_26589d8e-4d36-466c-993c-2d0fb23c2817/kube-rbac-proxy/0.log" Aug 04 17:00:29.971544 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:29.971515 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-scrxz_de8b7d3f-9598-47f7-a387-8117c80acc3c/dns-node-resolver/0.log" Aug 04 17:00:30.572078 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:30.572043 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ppxb_6484c73f-9461-43ec-bf2a-924665d325a6/node-ca/0.log" Aug 04 17:00:31.541303 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:31.541271 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k_8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227/istio-proxy/0.log" Aug 04 17:00:31.817374 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:31.817299 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7gqdr_9f3fb966-8f4b-4709-bbbf-1f45529c4eaf/istio-proxy/0.log" Aug 04 17:00:31.826128 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:31.826101 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7gqdr_9f3fb966-8f4b-4709-bbbf-1f45529c4eaf/istio-proxy/1.log" Aug 04 17:00:31.902694 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:31.902656 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6vftr_494ec520-130e-4ca2-a9ca-4f11a19f69eb/payload-processing/0.log" Aug 04 17:00:32.757333 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:32.757300 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vmbgf_69637f37-b07c-498f-9fbc-35d463e51e58/serve-healthcheck-canary/0.log" Aug 04 17:00:33.252907 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:33.252872 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4njrx_e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b/kube-rbac-proxy/0.log" Aug 04 17:00:33.278055 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:33.278027 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4njrx_e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b/exporter/0.log" Aug 04 17:00:33.300679 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:33.300648 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4njrx_e51c47b6-5f45-40a7-bfe0-1f2bca6e2a2b/extractor/0.log" Aug 04 17:00:35.107217 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:35.106135 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b2bcd/perf-node-gather-daemonset-jhvpm" Aug 04 17:00:35.563086 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:35.563052 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5z45r_18303a78-30fe-4a3d-bbc6-d30bc4604c51/manager/0.log" Aug 04 17:00:36.007108 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:36.007065 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d474f94c9-frkfm_d4792e00-f3e3-4ddf-9bf6-6e39924d9129/manager/0.log" Aug 04 17:00:36.316548 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:36.316455 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfb598967-fv7cz_114027f0-89cf-4672-9f5e-44d5dab1e346/manager/0.log" Aug 04 17:00:36.359783 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:36.359754 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfb598967-fv7cz_114027f0-89cf-4672-9f5e-44d5dab1e346/copy-manifests/0.log" Aug 04 17:00:37.592964 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:37.592921 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k_8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227/istio-proxy/0.log" Aug 04 17:00:37.820655 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:37.820603 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5967759b96-mbcvc_7a20b218-12f0-4e7b-ba34-579476907d4f/manager/0.log" Aug 04 17:00:37.822473 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:37.822437 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7gqdr_9f3fb966-8f4b-4709-bbbf-1f45529c4eaf/istio-proxy/0.log" Aug 04 17:00:37.832082 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:37.832056 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7gqdr_9f3fb966-8f4b-4709-bbbf-1f45529c4eaf/istio-proxy/1.log" Aug 04 17:00:37.888689 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:37.888608 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6vftr_494ec520-130e-4ca2-a9ca-4f11a19f69eb/payload-processing/0.log" Aug 04 17:00:39.301434 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:39.301407 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2zn5k_8b5cd07d-d8e3-45fa-9d54-b6e0fb5bc227/istio-proxy/0.log" Aug 04 17:00:39.545266 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:39.545231 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7gqdr_9f3fb966-8f4b-4709-bbbf-1f45529c4eaf/istio-proxy/0.log" Aug 04 17:00:39.556423 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:39.556322 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7gqdr_9f3fb966-8f4b-4709-bbbf-1f45529c4eaf/istio-proxy/1.log" Aug 04 17:00:39.612872 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:39.612789 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6vftr_494ec520-130e-4ca2-a9ca-4f11a19f69eb/payload-processing/0.log" Aug 04 17:00:44.490873 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.490842 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s8xlk_d05e8f95-eabc-4eb2-8f47-116955bc09a0/kube-multus-additional-cni-plugins/0.log" Aug 04 17:00:44.510827 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.510797 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s8xlk_d05e8f95-eabc-4eb2-8f47-116955bc09a0/egress-router-binary-copy/0.log" Aug 04 17:00:44.530024 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.529997 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s8xlk_d05e8f95-eabc-4eb2-8f47-116955bc09a0/cni-plugins/0.log" Aug 04 17:00:44.549158 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.549130 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s8xlk_d05e8f95-eabc-4eb2-8f47-116955bc09a0/bond-cni-plugin/0.log" Aug 04 17:00:44.569395 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.569364 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s8xlk_d05e8f95-eabc-4eb2-8f47-116955bc09a0/routeoverride-cni/0.log" Aug 04 17:00:44.588478 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.588443 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s8xlk_d05e8f95-eabc-4eb2-8f47-116955bc09a0/whereabouts-cni-bincopy/0.log" Aug 04 17:00:44.607411 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.607382 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s8xlk_d05e8f95-eabc-4eb2-8f47-116955bc09a0/whereabouts-cni/0.log" Aug 04 17:00:44.714132 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.714103 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vzlhw_77cfdd06-8d08-4813-848f-43139a3c05b7/kube-multus/0.log" Aug 04 17:00:44.778467 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.778395 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p6slt_4e47afd1-3679-47c0-bbdb-66ef0145e925/network-metrics-daemon/0.log" Aug 04 17:00:44.794200 ip-10-0-132-227 kubenswrapper[2556]: I0804 17:00:44.794173 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p6slt_4e47afd1-3679-47c0-bbdb-66ef0145e925/kube-rbac-proxy/0.log"