Sep 08 15:17:39.841733 ip-10-0-132-25 systemd[1]: Starting Kubernetes Kubelet... Sep 08 15:17:40.283229 ip-10-0-132-25 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 15:17:40.283229 ip-10-0-132-25 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 08 15:17:40.283229 ip-10-0-132-25 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 15:17:40.283229 ip-10-0-132-25 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 08 15:17:40.283229 ip-10-0-132-25 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 15:17:40.283938 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.283321 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 08 15:17:40.290663 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290634 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 15:17:40.290663 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290658 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 15:17:40.290663 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290663 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 15:17:40.290663 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290667 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 15:17:40.290663 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290670 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290676 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290680 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290684 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290687 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290690 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290693 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290696 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290698 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290701 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290704 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290707 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290710 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290712 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290715 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290717 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290720 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290722 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290725 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290729 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 15:17:40.290880 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290731 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290734 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290736 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290739 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290741 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290744 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290747 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290749 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290752 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290754 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290757 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290760 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290762 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290765 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290769 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290771 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290774 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290777 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290779 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290782 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 15:17:40.291354 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290785 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290788 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290790 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290792 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290795 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290797 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290803 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290805 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290808 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290810 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290813 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290815 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290820 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290823 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290826 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290829 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290832 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290835 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290837 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290840 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 15:17:40.291894 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290843 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290845 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290848 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290850 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290853 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290855 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290859 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290862 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290864 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290867 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290870 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290872 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290875 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290877 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290879 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290882 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290885 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290887 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290890 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 15:17:40.292394 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290893 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290896 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290898 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290901 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290905 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290908 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290910 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290914 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290919 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290923 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290925 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290928 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290931 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290933 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290936 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.290938 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291702 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291709 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291712 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 15:17:40.292852 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291717 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291719 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291722 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291725 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291727 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291730 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291733 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291735 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291738 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291741 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291743 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291746 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291748 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291751 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291754 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291757 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291760 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291762 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291764 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291767 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 15:17:40.293316 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291770 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291772 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291775 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291779 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291783 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291786 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291789 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291791 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291794 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291796 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291799 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291802 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291805 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291809 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291812 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291816 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291819 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291823 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 15:17:40.293837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291828 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291830 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291834 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291837 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291839 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291841 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291845 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291847 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291850 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291853 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291856 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291858 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291861 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291864 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291866 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291869 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291872 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291874 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291876 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291879 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 15:17:40.294300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291881 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291884 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291886 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291889 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291891 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291894 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291898 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291900 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291903 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291906 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291909 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291911 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291914 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291916 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291919 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291921 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291924 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291926 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291929 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291931 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 15:17:40.294800 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291934 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291936 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291939 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291942 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291944 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291947 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291949 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291952 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291954 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291957 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291959 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291962 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291964 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291967 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291969 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291971 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291974 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.291976 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292418 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292424 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 08 15:17:40.295281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292427 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292430 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292433 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292437 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292441 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292444 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292447 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292449 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292452 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292455 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292458 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292461 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292464 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292467 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292470 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292474 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292476 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292479 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292482 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292484 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 15:17:40.295862 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292487 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292489 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292492 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292494 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292496 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292499 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292501 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292504 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292506 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292508 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292513 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292516 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292518 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292521 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292523 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292526 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292529 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292531 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292533 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292536 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 15:17:40.296343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292538 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292541 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292543 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292546 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292548 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292553 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292555 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292558 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292561 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292564 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292566 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292569 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292571 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292575 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292578 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292581 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292583 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292586 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292589 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 15:17:40.296837 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292591 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292594 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292596 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292600 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292602 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292605 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292607 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292610 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292612 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292615 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292617 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292619 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292622 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292625 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292627 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292629 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292632 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292634 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292640 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292642 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 15:17:40.297300 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292645 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292647 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292650 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292653 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292655 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292658 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292662 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292665 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292668 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292671 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292673 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292676 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292679 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292682 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292685 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292690 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292693 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.292695 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292770 2578 flags.go:64] FLAG: --address="0.0.0.0" Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292809 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 08 15:17:40.297794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292815 2578 flags.go:64] FLAG: --anonymous-auth="true" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292820 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292825 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292828 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292832 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292836 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292840 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292843 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292846 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292849 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292854 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292857 2578 flags.go:64] FLAG: --cgroup-root="" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292860 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292863 2578 flags.go:64] FLAG: --client-ca-file="" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292867 2578 flags.go:64] FLAG: --cloud-provider="external" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292870 2578 flags.go:64] FLAG: --cluster-dns="[]" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292875 2578 flags.go:64] FLAG: --cluster-domain="" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292878 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292881 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292884 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292887 2578 flags.go:64] FLAG: --container-log-max-files="5" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292891 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292894 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 08 15:17:40.298273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292897 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292900 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292903 2578 flags.go:64] FLAG: --contention-profiling="false" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292906 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292910 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292913 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292916 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292920 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292923 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292926 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292929 2578 flags.go:64] FLAG: --enable-load-reader="false" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292932 2578 flags.go:64] FLAG: --enable-server="true" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292935 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292939 2578 flags.go:64] FLAG: --event-burst="100" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292942 2578 flags.go:64] FLAG: --event-qps="50" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292945 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292947 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292950 2578 flags.go:64] FLAG: --eviction-hard="" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292963 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292967 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292970 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292973 2578 flags.go:64] FLAG: --eviction-soft="" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292975 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292978 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292982 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 08 15:17:40.298829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292985 2578 flags.go:64] FLAG: --experimental-mounter-path="" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292987 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292990 2578 flags.go:64] FLAG: --fail-swap-on="true" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292993 2578 flags.go:64] FLAG: --feature-gates="" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292996 2578 flags.go:64] FLAG: --file-check-frequency="20s" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.292999 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293002 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293005 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293008 2578 flags.go:64] FLAG: --healthz-port="10248" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293011 2578 flags.go:64] FLAG: --help="false" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293014 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293019 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293021 2578 flags.go:64] FLAG: --http-check-frequency="20s" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293024 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293027 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293031 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293033 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293036 2578 flags.go:64] FLAG: --image-service-endpoint="" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293039 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293041 2578 flags.go:64] FLAG: --kube-api-burst="100" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293044 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293047 2578 flags.go:64] FLAG: --kube-api-qps="50" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293050 2578 flags.go:64] FLAG: --kube-reserved="" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293052 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 08 15:17:40.299502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293055 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293058 2578 flags.go:64] FLAG: --kubelet-cgroups="" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293061 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293063 2578 flags.go:64] FLAG: --lock-file="" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293066 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293069 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293071 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293077 2578 flags.go:64] FLAG: --log-json-split-stream="false" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293080 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293083 2578 flags.go:64] FLAG: --log-text-split-stream="false" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293085 2578 flags.go:64] FLAG: --logging-format="text" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293088 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293091 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293094 2578 flags.go:64] FLAG: --manifest-url="" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293097 2578 flags.go:64] FLAG: --manifest-url-header="" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293101 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293104 2578 flags.go:64] FLAG: --max-open-files="1000000" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293108 2578 flags.go:64] FLAG: --max-pods="110" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293110 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293115 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293117 2578 flags.go:64] FLAG: --memory-manager-policy="None" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293120 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293123 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293126 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293128 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 08 15:17:40.300086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293136 2578 flags.go:64] FLAG: --node-status-max-images="50" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293139 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293141 2578 flags.go:64] FLAG: --oom-score-adj="-999" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293144 2578 flags.go:64] FLAG: --pod-cidr="" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293147 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293153 2578 flags.go:64] FLAG: --pod-manifest-path="" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293156 2578 flags.go:64] FLAG: --pod-max-pids="-1" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293159 2578 flags.go:64] FLAG: --pods-per-core="0" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293161 2578 flags.go:64] FLAG: --port="10250" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293165 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293168 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04cbedd522cf1ef46" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293171 2578 flags.go:64] FLAG: --qos-reserved="" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293174 2578 flags.go:64] FLAG: --read-only-port="10255" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293177 2578 flags.go:64] FLAG: --register-node="true" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293179 2578 flags.go:64] FLAG: --register-with-taints="" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293183 2578 flags.go:64] FLAG: --registry-burst="10" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293186 2578 flags.go:64] FLAG: --registry-qps="5" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293189 2578 flags.go:64] FLAG: --reserved-cpus="" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293191 2578 flags.go:64] FLAG: --reserved-memory="" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293195 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293198 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293200 2578 flags.go:64] FLAG: --rotate-certificates="false" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293203 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293206 2578 flags.go:64] FLAG: --runonce="false" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293209 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 08 15:17:40.300707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293211 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293215 2578 flags.go:64] FLAG: --seccomp-default="false" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293219 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293222 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293225 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293228 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293231 2578 flags.go:64] FLAG: --storage-driver-password="root" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293234 2578 flags.go:64] FLAG: --storage-driver-secure="false" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293237 2578 flags.go:64] FLAG: --storage-driver-table="stats" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293239 2578 flags.go:64] FLAG: --storage-driver-user="root" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293242 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293245 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293248 2578 flags.go:64] FLAG: --system-cgroups="" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293250 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293255 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293258 2578 flags.go:64] FLAG: --tls-cert-file="" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293261 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293266 2578 flags.go:64] FLAG: --tls-min-version="" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293269 2578 flags.go:64] FLAG: --tls-private-key-file="" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293271 2578 flags.go:64] FLAG: --topology-manager-policy="none" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293274 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293277 2578 flags.go:64] FLAG: --topology-manager-scope="container" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293280 2578 flags.go:64] FLAG: --v="2" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293283 2578 flags.go:64] FLAG: --version="false" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293288 2578 flags.go:64] FLAG: --vmodule="" Sep 08 15:17:40.301330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293292 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.293295 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293421 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293425 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293428 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293431 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293433 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293436 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293438 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293441 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293444 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293447 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293449 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293452 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293454 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293457 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293459 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293478 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293483 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293486 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 15:17:40.301959 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293489 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293492 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293494 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293498 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293501 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293503 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293506 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293508 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293511 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293513 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293516 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293518 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293521 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293524 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293526 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293529 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293532 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293534 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293536 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293539 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 15:17:40.302521 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293542 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293545 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293548 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293550 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293553 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293555 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293558 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293560 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293563 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293565 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293568 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293570 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293574 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293578 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293580 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293584 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293587 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293590 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293592 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 15:17:40.303000 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293595 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293598 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293600 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293603 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293605 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293608 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293612 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293615 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293617 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293620 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293622 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293625 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293627 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293629 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293633 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293635 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293638 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293640 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293643 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293645 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 15:17:40.303487 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293648 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293650 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293653 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293655 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293658 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293661 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293663 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293667 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293672 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293675 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293677 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293681 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293684 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293687 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293690 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293693 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293696 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293698 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293701 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 15:17:40.303974 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293704 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293707 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.293710 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.294475 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.302547 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.302564 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.302585 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.302591 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302650 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302655 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302658 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302661 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302664 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302667 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302670 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 15:17:40.304440 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302672 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302675 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302678 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302680 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302684 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302689 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302692 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302695 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302698 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302701 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302703 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302706 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302709 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302712 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302714 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302717 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302719 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302722 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302725 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302727 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 15:17:40.304860 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302730 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302732 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302737 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302740 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302742 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302745 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302747 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302750 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302752 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302755 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302757 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302760 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302762 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302765 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302767 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302769 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302772 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302774 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302777 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302779 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 15:17:40.305393 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302782 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302785 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302788 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302792 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302795 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302797 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302800 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302802 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302805 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302807 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302810 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302812 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302815 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302817 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302821 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302824 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302827 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302829 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302832 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 15:17:40.305896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302834 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302837 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302839 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302842 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302846 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302850 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302852 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302855 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302858 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302860 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302863 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302865 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302867 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302870 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302872 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302875 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302877 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302879 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302882 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302884 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 15:17:40.306358 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302887 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302889 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302892 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302894 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302896 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302899 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302902 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302905 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302908 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302911 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302914 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302917 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.302919 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.302924 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 15:17:40.306936 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303037 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303043 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303046 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303049 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303052 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303054 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303057 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303060 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303064 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303068 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303070 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303073 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303076 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303078 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303081 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303083 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303085 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303088 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303090 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 15:17:40.307281 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303093 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303095 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303098 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303100 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303103 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303105 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303107 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303111 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303113 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303116 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303119 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303122 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303124 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303126 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303129 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303132 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303134 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303136 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303139 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303142 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 15:17:40.307747 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303144 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303146 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303149 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303152 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303154 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303156 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303159 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303161 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303164 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303166 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303168 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303172 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303175 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303178 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303181 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303183 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303185 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303188 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303190 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 15:17:40.308233 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303193 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303196 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303198 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303201 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303203 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303206 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303209 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303211 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303214 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303216 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303218 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303221 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303224 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303226 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303229 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303231 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303234 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303236 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303239 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303241 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 15:17:40.308697 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303243 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303246 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303249 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303252 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303255 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303258 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303260 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303263 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303265 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303268 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303270 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303272 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303275 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303278 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303280 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303282 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303285 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303288 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303291 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303293 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 15:17:40.309176 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:40.303295 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 15:17:40.309679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.303301 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 15:17:40.309679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.303458 2578 server.go:962] "Client rotation is on, will bootstrap in background" Sep 08 15:17:40.309679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.305931 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 08 15:17:40.309679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.306996 2578 server.go:1019] "Starting client certificate rotation" Sep 08 15:17:40.309679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.307093 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 15:17:40.309679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.307133 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 15:17:40.309924 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.309909 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 15:17:40.311067 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.311049 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 15:17:40.322639 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.322619 2578 log.go:25] "Validated CRI v1 runtime API" Sep 08 15:17:40.325825 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.325808 2578 log.go:25] "Validated CRI v1 image API" Sep 08 15:17:40.326703 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.326687 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 08 15:17:40.329358 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.329333 2578 fs.go:135] Filesystem UUIDs: map[730c1a28-822d-47af-93b0-985b37f93fb6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b04a22cb-54e5-4c81-bffa-739bd260b3b5:/dev/nvme0n1p4] Sep 08 15:17:40.329358 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.329352 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 08 15:17:40.334367 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.334249 2578 manager.go:216] Machine: {Timestamp:2026-09-08 15:17:40.332919132 +0000 UTC m=+0.378280977 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099984 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2140d93637f755bb2c8d571943b175 SystemUUID:ec2140d9-3637-f755-bb2c-8d571943b175 BootID:061ee426-bdb8-4ee0-8574-49072188912e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a3:68:99:64:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a3:68:99:64:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:8e:bd:0e:eb:b3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 08 15:17:40.334367 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.334358 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 08 15:17:40.334479 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.334449 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 08 15:17:40.338045 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.338017 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 08 15:17:40.338189 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.338049 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-25.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 08 15:17:40.338901 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.338885 2578 topology_manager.go:138] "Creating topology manager with none policy" Sep 08 15:17:40.338961 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.338908 2578 container_manager_linux.go:309] "Creating device plugin manager" Sep 08 15:17:40.338961 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.338936 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 15:17:40.339871 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.339857 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 15:17:40.341753 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.341742 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 08 15:17:40.342049 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.342029 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 15:17:40.346535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.346521 2578 state_mem.go:36] "Initialized new in-memory state store" Sep 08 15:17:40.346660 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.346650 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 08 15:17:40.346702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.346683 2578 kubelet.go:486] "Attempting to sync node with API server" Sep 08 15:17:40.346702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.346694 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 08 15:17:40.346791 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.346714 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 08 15:17:40.346791 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.346724 2578 kubelet.go:398] "Adding apiserver pod source" Sep 08 15:17:40.346791 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.346734 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 08 15:17:40.347089 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.347078 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 15:17:40.347124 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.347097 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 15:17:40.349515 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.349500 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 08 15:17:40.352424 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.352407 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 08 15:17:40.352476 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.352440 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 08 15:17:40.352653 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.352640 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 08 15:17:40.352686 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.352663 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 08 15:17:40.352686 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.352672 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 08 15:17:40.352686 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.352678 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 08 15:17:40.352686 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.352687 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 08 15:17:40.352789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.352693 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 08 15:17:40.353278 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.353268 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 08 15:17:40.353278 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.353278 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 08 15:17:40.353338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.353284 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 08 15:17:40.353338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.353296 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 08 15:17:40.353338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.353311 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 08 15:17:40.353338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.353319 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 08 15:17:40.354172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.354161 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 08 15:17:40.354172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.354172 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 08 15:17:40.355663 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.355635 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-25.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 08 15:17:40.355780 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.355758 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-25.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 08 15:17:40.355856 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.355763 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 08 15:17:40.358543 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.358320 2578 server.go:1268] "Started kubelet" Sep 08 15:17:40.358644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.358465 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 08 15:17:40.358644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.358434 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 08 15:17:40.358644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.358636 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 08 15:17:40.363183 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.363162 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 08 15:17:40.363634 ip-10-0-132-25 systemd[1]: Started Kubernetes Kubelet. Sep 08 15:17:40.365108 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.365076 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 08 15:17:40.365443 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.365422 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bt2vd" Sep 08 15:17:40.366351 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.366331 2578 server.go:310] "Adding debug handlers to kubelet server" Sep 08 15:17:40.368490 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.367347 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-25.ec2.internal.18d3617b9238c955 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-25.ec2.internal,UID:ip-10-0-132-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-25.ec2.internal,},FirstTimestamp:2026-09-08 15:17:40.358285653 +0000 UTC m=+0.403647497,LastTimestamp:2026-09-08 15:17:40.358285653 +0000 UTC m=+0.403647497,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-25.ec2.internal,}" Sep 08 15:17:40.371182 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.371163 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 08 15:17:40.371894 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.371879 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 08 15:17:40.371997 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.371975 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:40.372816 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.372798 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 08 15:17:40.372816 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.372801 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 08 15:17:40.372928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.372828 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 08 15:17:40.372928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.372908 2578 reconstruct.go:97] "Volume reconstruction finished" Sep 08 15:17:40.372928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.372915 2578 reconciler.go:29] "Reconciler: start to sync state" Sep 08 15:17:40.373052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.372974 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 08 15:17:40.373052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.372988 2578 factory.go:55] Registering systemd factory Sep 08 15:17:40.373052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.372996 2578 factory.go:223] Registration of the systemd container factory successfully Sep 08 15:17:40.373254 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.373242 2578 factory.go:153] Registering CRI-O factory Sep 08 15:17:40.373287 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.373256 2578 factory.go:223] Registration of the crio container factory successfully Sep 08 15:17:40.373287 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.373269 2578 factory.go:103] Registering Raw factory Sep 08 15:17:40.373287 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.373281 2578 manager.go:1196] Started watching for new ooms in manager Sep 08 15:17:40.373939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.373920 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bt2vd" Sep 08 15:17:40.374300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.374283 2578 manager.go:349] Starting recovery of all containers Sep 08 15:17:40.381243 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.381207 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 08 15:17:40.383144 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.383123 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 15:17:40.385517 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.385501 2578 manager.go:354] Recovery completed Sep 08 15:17:40.387018 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.387000 2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-25.ec2.internal\" not found" node="ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.389822 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.389807 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 15:17:40.392015 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.391997 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientMemory" Sep 08 15:17:40.392069 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.392029 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 15:17:40.392069 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.392039 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientPID" Sep 08 15:17:40.392624 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.392613 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 08 15:17:40.392624 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.392621 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 08 15:17:40.392624 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.392626 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 15:17:40.392767 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.392637 2578 state_mem.go:36] "Initialized new in-memory state store" Sep 08 15:17:40.393281 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.393267 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientMemory" Sep 08 15:17:40.393338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.393296 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 15:17:40.393338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.393310 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientPID" Sep 08 15:17:40.394722 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.394710 2578 policy_none.go:49] "None policy: Start" Sep 08 15:17:40.394773 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.394726 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 08 15:17:40.394773 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.394735 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 08 15:17:40.396745 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.396735 2578 policy_none.go:47] "Start" Sep 08 15:17:40.435177 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.435152 2578 manager.go:337] "Starting Device Plugin manager" Sep 08 15:17:40.455019 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.435235 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 08 15:17:40.455019 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.435248 2578 server.go:87] "Starting device plugin registration server" Sep 08 15:17:40.455019 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.435682 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 08 15:17:40.455019 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.435694 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 08 15:17:40.455019 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.435794 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 08 15:17:40.455019 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.435883 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 08 15:17:40.455019 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.435895 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 08 15:17:40.455019 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.436492 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 08 15:17:40.455019 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.436549 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:40.472966 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.472931 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 08 15:17:40.473106 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.472997 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 08 15:17:40.473106 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.473021 2578 kubelet.go:2448] "Starting kubelet main sync loop" Sep 08 15:17:40.473106 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.473067 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 08 15:17:40.475759 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.475740 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 15:17:40.536517 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.536416 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 15:17:40.538050 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.538029 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientMemory" Sep 08 15:17:40.538172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.538064 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 15:17:40.538172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.538078 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientPID" Sep 08 15:17:40.538172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.538100 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.547041 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.547018 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.547102 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.547049 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-25.ec2.internal\": node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:40.568007 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.567975 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:40.574102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.574078 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal"] Sep 08 15:17:40.574156 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.574150 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 15:17:40.575745 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.575727 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientMemory" Sep 08 15:17:40.575833 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.575764 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 15:17:40.575833 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.575780 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientPID" Sep 08 15:17:40.578078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.578065 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 15:17:40.578199 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.578184 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.578241 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.578216 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 15:17:40.578928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.578910 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientMemory" Sep 08 15:17:40.578928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.578931 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 15:17:40.579061 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.578941 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientPID" Sep 08 15:17:40.579061 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.578959 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientMemory" Sep 08 15:17:40.579061 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.578981 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 15:17:40.579061 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.578992 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientPID" Sep 08 15:17:40.581172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.581154 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.581269 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.581178 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 15:17:40.581891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.581878 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientMemory" Sep 08 15:17:40.581950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.581903 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 15:17:40.581950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.581913 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeHasSufficientPID" Sep 08 15:17:40.620196 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.620165 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-25.ec2.internal\" not found" node="ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.624827 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.624807 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-25.ec2.internal\" not found" node="ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.668639 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.668605 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:40.769439 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.769408 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:40.774766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.774744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d1459c4d11b8240fe56af12f6a1ae42-config\") pod \"kube-apiserver-proxy-ip-10-0-132-25.ec2.internal\" (UID: \"4d1459c4d11b8240fe56af12f6a1ae42\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.774832 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.774774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4efd810d1fa9f921edf1aac1347d9cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal\" (UID: \"d4efd810d1fa9f921edf1aac1347d9cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.774832 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.774793 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4efd810d1fa9f921edf1aac1347d9cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal\" (UID: \"d4efd810d1fa9f921edf1aac1347d9cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.870277 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.870199 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:40.875640 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.875606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d1459c4d11b8240fe56af12f6a1ae42-config\") pod \"kube-apiserver-proxy-ip-10-0-132-25.ec2.internal\" (UID: \"4d1459c4d11b8240fe56af12f6a1ae42\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.875691 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.875646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4efd810d1fa9f921edf1aac1347d9cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal\" (UID: \"d4efd810d1fa9f921edf1aac1347d9cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.875691 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.875669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4efd810d1fa9f921edf1aac1347d9cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal\" (UID: \"d4efd810d1fa9f921edf1aac1347d9cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.875750 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.875735 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4efd810d1fa9f921edf1aac1347d9cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal\" (UID: \"d4efd810d1fa9f921edf1aac1347d9cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.875750 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.875740 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d1459c4d11b8240fe56af12f6a1ae42-config\") pod \"kube-apiserver-proxy-ip-10-0-132-25.ec2.internal\" (UID: \"4d1459c4d11b8240fe56af12f6a1ae42\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.875814 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.875749 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4efd810d1fa9f921edf1aac1347d9cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal\" (UID: \"d4efd810d1fa9f921edf1aac1347d9cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.922775 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.922732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.928105 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:40.927961 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal" Sep 08 15:17:40.970705 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:40.970673 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:41.071248 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:41.071220 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:41.171781 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:41.171709 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:41.272245 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:41.272202 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:41.307659 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:41.307633 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 08 15:17:41.308089 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:41.307848 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 15:17:41.308089 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:41.307846 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 15:17:41.372412 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:41.372367 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:41.378184 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:41.378158 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 15:12:41 +0000 UTC" deadline="2028-05-27 08:27:19.465102048 +0000 UTC" Sep 08 15:17:41.378271 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:41.378185 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15041h9m38.086920745s" Sep 08 15:17:41.473151 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:41.473076 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-25.ec2.internal\" not found" Sep 08 15:17:41.507589 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:41.507558 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 15:17:41.572531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:41.572498 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" Sep 08 15:17:41.574028 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:41.574004 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 15:17:42.606265 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:42.606250 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 15:17:42.607285 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:42.607271 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal" Sep 08 15:17:42.617430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:42.617413 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 15:17:42.662527 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:42.662510 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 15:17:43.166269 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.166240 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 15:17:43.369520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.369490 2578 apiserver.go:52] "Watching apiserver" Sep 08 15:17:43.377720 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.377695 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 08 15:17:43.380052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.378544 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-hdcjp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf","openshift-cluster-node-tuning-operator/tuned-9mz5g","openshift-dns/node-resolver-lx75p","openshift-multus/multus-additional-cni-plugins-ngkcc","openshift-multus/multus-jsck4","openshift-ovn-kubernetes/ovnkube-node-vkqdz","kube-system/global-pull-secret-syncer-b6g8j","kube-system/konnectivity-agent-vt2b6","kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal","openshift-image-registry/node-ca-zmlkq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal","openshift-multus/network-metrics-daemon-c6lrg","openshift-network-diagnostics/network-check-target-c975x"] Sep 08 15:17:43.381251 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.381231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.383731 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.383697 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:43.383844 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.383791 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:17:43.384214 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.384194 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 08 15:17:43.384366 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.384202 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 08 15:17:43.384501 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.384483 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 08 15:17:43.384580 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.384546 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 08 15:17:43.384629 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.384582 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 08 15:17:43.384680 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.384666 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mzqpl\"" Sep 08 15:17:43.386214 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.386195 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 08 15:17:43.386471 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.386454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.388881 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.388859 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.389195 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.389170 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7nkfn\"" Sep 08 15:17:43.389432 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.389415 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 08 15:17:43.389524 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.389440 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 08 15:17:43.391098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.391077 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.391270 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.391256 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 08 15:17:43.391648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.391628 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 08 15:17:43.391843 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.391826 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g4nlv\"" Sep 08 15:17:43.393090 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.393073 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 08 15:17:43.393450 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.393435 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.394730 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.394626 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 08 15:17:43.394897 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.394879 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 08 15:17:43.395150 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.395132 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 08 15:17:43.395468 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.395448 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 08 15:17:43.395582 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.395563 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5vf69\"" Sep 08 15:17:43.395720 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.395704 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 08 15:17:43.396102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.396084 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:17:43.396521 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.396504 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-67dtq\"" Sep 08 15:17:43.396861 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.396686 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 08 15:17:43.398553 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.398534 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.398709 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.398683 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 08 15:17:43.398789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.398699 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-x5ww6\"" Sep 08 15:17:43.399012 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.398994 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 08 15:17:43.401083 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.401062 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-85dt6\"" Sep 08 15:17:43.401563 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.401546 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 08 15:17:43.401659 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.401646 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 08 15:17:43.402219 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.402202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.402746 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.402554 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 08 15:17:43.404389 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404369 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-kubelet\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.404498 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-run-systemd\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.404498 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404430 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-etc-openvswitch\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.404498 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.404651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404560 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0f26d665-ebd6-4939-861f-6085d394f06b-hosts-file\") pod \"node-resolver-lx75p\" (UID: \"0f26d665-ebd6-4939-861f-6085d394f06b\") " pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.404651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-systemd\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.404651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404619 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-run\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.404651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404641 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:43.404838 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404641 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdqxh\" (UniqueName: \"kubernetes.io/projected/63f55018-af6a-4631-8222-0a268c900911-kube-api-access-xdqxh\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:43.404838 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.404735 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:17:43.404838 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svg7t\" (UniqueName: \"kubernetes.io/projected/78d7df79-1429-47c1-9358-92675a95aa9f-kube-api-access-svg7t\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.405098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404846 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.405098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-cni-dir\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.405098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404928 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c6hlm\"" Sep 08 15:17:43.405098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404997 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 08 15:17:43.405098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.404969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-daemon-config\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.405098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-etc-kubernetes\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.405098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405082 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f772ce72-534a-48c0-88a1-de406f5eb5ca-ovn-node-metrics-cert\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-cni-netd\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405261 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f772ce72-534a-48c0-88a1-de406f5eb5ca-env-overrides\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405285 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78d7df79-1429-47c1-9358-92675a95aa9f-tmp\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405310 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-run-netns\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405335 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-socket-dir-parent\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405386 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405394 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-system-cni-dir\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-sysctl-conf\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-sys\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.405472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405474 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-cnibin\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.405859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t42m9\" (UniqueName: \"kubernetes.io/projected/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-kube-api-access-t42m9\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.405859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405517 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/92a884a8-1be4-4cfa-bc2e-21dbcc43b79a-agent-certs\") pod \"konnectivity-agent-vt2b6\" (UID: \"92a884a8-1be4-4cfa-bc2e-21dbcc43b79a\") " pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:17:43.405859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/92a884a8-1be4-4cfa-bc2e-21dbcc43b79a-konnectivity-ca\") pod \"konnectivity-agent-vt2b6\" (UID: \"92a884a8-1be4-4cfa-bc2e-21dbcc43b79a\") " pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:17:43.405859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405558 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f772ce72-534a-48c0-88a1-de406f5eb5ca-ovnkube-script-lib\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.405859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-modprobe-d\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.405859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-host\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.405859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-run-netns\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.405859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405813 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-run-ovn\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.405859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405826 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-run-ovn-kubernetes\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-var-lib-kubelet\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405900 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405925 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-run-openvswitch\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-lib-modules\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.405980 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-hostroot\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fglm9\" (UniqueName: \"kubernetes.io/projected/6d13a333-3ca3-4724-b59b-668f9b451da2-kube-api-access-fglm9\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406045 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-node-log\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406090 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f772ce72-534a-48c0-88a1-de406f5eb5ca-ovnkube-config\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406194 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-system-cni-dir\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-cni-binary-copy\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-run-k8s-cni-cncf-io\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406279 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-cnibin\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.406300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406300 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-var-lib-cni-bin\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.406795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-var-lib-kubelet\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.406795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-systemd-units\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.406795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-run-multus-certs\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.406795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-slash\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.406795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.406466 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz7vr\" (UniqueName: \"kubernetes.io/projected/f772ce72-534a-48c0-88a1-de406f5eb5ca-kube-api-access-nz7vr\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.407527 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407481 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f26d665-ebd6-4939-861f-6085d394f06b-tmp-dir\") pod \"node-resolver-lx75p\" (UID: \"0f26d665-ebd6-4939-861f-6085d394f06b\") " pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.407619 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407568 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p278z\" (UniqueName: \"kubernetes.io/projected/0f26d665-ebd6-4939-861f-6085d394f06b-kube-api-access-p278z\") pod \"node-resolver-lx75p\" (UID: \"0f26d665-ebd6-4939-861f-6085d394f06b\") " pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.407707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/78d7df79-1429-47c1-9358-92675a95aa9f-etc-tuned\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.407756 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407732 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-var-lib-cni-multus\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.407806 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-log-socket\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.407919 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407905 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-var-lib-openvswitch\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.407956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-kubernetes\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.407995 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407947 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 15:17:43.407995 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407958 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-sysctl-d\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.408060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.407988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-conf-dir\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.408060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.408029 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-cni-bin\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.408150 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.408057 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-sysconfig\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.408150 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.408095 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.408150 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.408141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d13a333-3ca3-4724-b59b-668f9b451da2-cni-binary-copy\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.409138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.409101 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.411521 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.411483 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:43.411629 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.411583 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:17:43.413681 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.413663 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 08 15:17:43.413775 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.413733 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z55zc\"" Sep 08 15:17:43.414829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.414649 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 08 15:17:43.414829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.414719 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 08 15:17:43.428587 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.428565 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qrq98" Sep 08 15:17:43.436713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.436690 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qrq98" Sep 08 15:17:43.471572 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.471535 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 15:17:43.495981 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.495957 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 08 15:17:43.498996 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.498939 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal" event={"ID":"4d1459c4d11b8240fe56af12f6a1ae42","Type":"ContainerStarted","Data":"97ca05a6f81888e9cd563213e3acb999dc8333f71ba8df120e56fa8646f123cb"} Sep 08 15:17:43.508984 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.508958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-cni-netd\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.509102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.508996 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f772ce72-534a-48c0-88a1-de406f5eb5ca-env-overrides\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.509102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78d7df79-1429-47c1-9358-92675a95aa9f-tmp\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.509102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509044 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-run-netns\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.509102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-socket-dir-parent\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.509317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509123 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:43.509317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk8jz\" (UniqueName: \"kubernetes.io/projected/d3a52f97-efe0-4757-b52b-9a1bd61d48c0-kube-api-access-qk8jz\") pod \"iptables-alerter-hdcjp\" (UID: \"d3a52f97-efe0-4757-b52b-9a1bd61d48c0\") " pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.509317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509190 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-sys-fs\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.509317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509225 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1baffed3-ce48-4637-abcd-01dd21feed17-kubelet-config\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:43.509317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509249 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-system-cni-dir\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.509317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-sysctl-conf\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.509317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-sys\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-cnibin\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509349 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t42m9\" (UniqueName: \"kubernetes.io/projected/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-kube-api-access-t42m9\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509372 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/92a884a8-1be4-4cfa-bc2e-21dbcc43b79a-agent-certs\") pod \"konnectivity-agent-vt2b6\" (UID: \"92a884a8-1be4-4cfa-bc2e-21dbcc43b79a\") " pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509406 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/92a884a8-1be4-4cfa-bc2e-21dbcc43b79a-konnectivity-ca\") pod \"konnectivity-agent-vt2b6\" (UID: \"92a884a8-1be4-4cfa-bc2e-21dbcc43b79a\") " pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509428 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f772ce72-534a-48c0-88a1-de406f5eb5ca-ovnkube-script-lib\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509478 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-modprobe-d\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509499 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-host\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-run-netns\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-run-ovn\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509582 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-run-ovn-kubernetes\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-var-lib-kubelet\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509624 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f772ce72-534a-48c0-88a1-de406f5eb5ca-env-overrides\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.509648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509629 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-run-openvswitch\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-lib-modules\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509721 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-run-netns\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-hostroot\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509734 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fglm9\" (UniqueName: \"kubernetes.io/projected/6d13a333-3ca3-4724-b59b-668f9b451da2-kube-api-access-fglm9\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1baffed3-ce48-4637-abcd-01dd21feed17-dbus\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-node-log\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f772ce72-534a-48c0-88a1-de406f5eb5ca-ovnkube-config\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509901 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-system-cni-dir\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-cni-binary-copy\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-run-k8s-cni-cncf-io\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3a52f97-efe0-4757-b52b-9a1bd61d48c0-iptables-alerter-script\") pod \"iptables-alerter-hdcjp\" (UID: \"d3a52f97-efe0-4757-b52b-9a1bd61d48c0\") " pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-cnibin\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.510314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctbmg\" (UniqueName: \"kubernetes.io/projected/2254f035-44a3-40c2-8c27-fd9f12da4c69-kube-api-access-ctbmg\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510216 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-system-cni-dir\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509772 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-hostroot\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510259 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-socket-dir-parent\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.509078 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-cni-netd\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510296 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-node-log\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.510446 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510544 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-system-cni-dir\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.510606 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs podName:63f55018-af6a-4631-8222-0a268c900911 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:44.010566585 +0000 UTC m=+3.034191018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs") pod "network-metrics-daemon-c6lrg" (UID: "63f55018-af6a-4631-8222-0a268c900911") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510651 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-sysctl-conf\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510754 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-sys\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510799 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-cnibin\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510871 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-run-k8s-cni-cncf-io\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.511078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.510935 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-run-ovn\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-var-lib-cni-bin\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511179 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-var-lib-kubelet\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-systemd-units\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-run-multus-certs\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-registration-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-slash\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511348 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-cni-binary-copy\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511353 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-modprobe-d\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511352 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz7vr\" (UniqueName: \"kubernetes.io/projected/f772ce72-534a-48c0-88a1-de406f5eb5ca-kube-api-access-nz7vr\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511405 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f26d665-ebd6-4939-861f-6085d394f06b-tmp-dir\") pod \"node-resolver-lx75p\" (UID: \"0f26d665-ebd6-4939-861f-6085d394f06b\") " pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511421 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-host\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p278z\" (UniqueName: \"kubernetes.io/projected/0f26d665-ebd6-4939-861f-6085d394f06b-kube-api-access-p278z\") pod \"node-resolver-lx75p\" (UID: \"0f26d665-ebd6-4939-861f-6085d394f06b\") " pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511453 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/78d7df79-1429-47c1-9358-92675a95aa9f-etc-tuned\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511472 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-var-lib-cni-multus\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511495 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-socket-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.511752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511510 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-var-lib-kubelet\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511516 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-log-socket\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511531 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/92a884a8-1be4-4cfa-bc2e-21dbcc43b79a-konnectivity-ca\") pod \"konnectivity-agent-vt2b6\" (UID: \"92a884a8-1be4-4cfa-bc2e-21dbcc43b79a\") " pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-log-socket\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-var-lib-openvswitch\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-kubernetes\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511606 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-var-lib-openvswitch\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511596 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-run-ovn-kubernetes\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-sysctl-d\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511681 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-var-lib-kubelet\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511689 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-cnibin\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511732 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-run-multus-certs\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511752 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-var-lib-cni-bin\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511731 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-conf-dir\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511797 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-slash\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511831 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-cni-bin\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511469 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-run-netns\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.512547 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.511965 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-lib-modules\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512024 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-run-openvswitch\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512041 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f26d665-ebd6-4939-861f-6085d394f06b-tmp-dir\") pod \"node-resolver-lx75p\" (UID: \"0f26d665-ebd6-4939-861f-6085d394f06b\") " pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512049 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-sysconfig\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512064 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-kubernetes\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512094 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-host-var-lib-cni-multus\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512135 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-conf-dir\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512143 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d13a333-3ca3-4724-b59b-668f9b451da2-cni-binary-copy\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512159 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-cni-bin\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512163 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-sysctl-d\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512201 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-systemd-units\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512242 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-sysconfig\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-kubelet\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-run-systemd\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512343 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-etc-openvswitch\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512416 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0f26d665-ebd6-4939-861f-6085d394f06b-hosts-file\") pod \"node-resolver-lx75p\" (UID: \"0f26d665-ebd6-4939-861f-6085d394f06b\") " pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.513398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512443 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-systemd\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512465 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-run\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqxh\" (UniqueName: \"kubernetes.io/projected/63f55018-af6a-4631-8222-0a268c900911-kube-api-access-xdqxh\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512558 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svg7t\" (UniqueName: \"kubernetes.io/projected/78d7df79-1429-47c1-9358-92675a95aa9f-kube-api-access-svg7t\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512617 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-cni-dir\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512645 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-daemon-config\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-etc-kubernetes\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512673 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-run-systemd\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3a52f97-efe0-4757-b52b-9a1bd61d48c0-host-slash\") pod \"iptables-alerter-hdcjp\" (UID: \"d3a52f97-efe0-4757-b52b-9a1bd61d48c0\") " pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512770 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-etc-kubernetes\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512782 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512787 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-run\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512809 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d13a333-3ca3-4724-b59b-668f9b451da2-cni-binary-copy\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512850 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-etc-openvswitch\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512897 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-kubelet\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512934 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-device-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.514261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.512964 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f772ce72-534a-48c0-88a1-de406f5eb5ca-ovn-node-metrics-cert\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.513013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efd21b96-4ff3-467f-89dd-f1ea27040971-host\") pod \"node-ca-zmlkq\" (UID: \"efd21b96-4ff3-467f-89dd-f1ea27040971\") " pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.513037 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/efd21b96-4ff3-467f-89dd-f1ea27040971-serviceca\") pod \"node-ca-zmlkq\" (UID: \"efd21b96-4ff3-467f-89dd-f1ea27040971\") " pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.513097 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f772ce72-534a-48c0-88a1-de406f5eb5ca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.513188 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0f26d665-ebd6-4939-861f-6085d394f06b-hosts-file\") pod \"node-resolver-lx75p\" (UID: \"0f26d665-ebd6-4939-861f-6085d394f06b\") " pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.513191 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/78d7df79-1429-47c1-9358-92675a95aa9f-etc-systemd\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.513234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh9zg\" (UniqueName: \"kubernetes.io/projected/efd21b96-4ff3-467f-89dd-f1ea27040971-kube-api-access-mh9zg\") pod \"node-ca-zmlkq\" (UID: \"efd21b96-4ff3-467f-89dd-f1ea27040971\") " pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.513370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.513354 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-cni-dir\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.513470 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d13a333-3ca3-4724-b59b-668f9b451da2-multus-daemon-config\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.514366 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f772ce72-534a-48c0-88a1-de406f5eb5ca-ovnkube-config\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.514809 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/78d7df79-1429-47c1-9358-92675a95aa9f-etc-tuned\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.514857 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f772ce72-534a-48c0-88a1-de406f5eb5ca-ovnkube-script-lib\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.515066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.515068 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/92a884a8-1be4-4cfa-bc2e-21dbcc43b79a-agent-certs\") pod \"konnectivity-agent-vt2b6\" (UID: \"92a884a8-1be4-4cfa-bc2e-21dbcc43b79a\") " pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:17:43.515757 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.515735 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78d7df79-1429-47c1-9358-92675a95aa9f-tmp\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.516772 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.516753 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f772ce72-534a-48c0-88a1-de406f5eb5ca-ovn-node-metrics-cert\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.524804 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.524775 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fglm9\" (UniqueName: \"kubernetes.io/projected/6d13a333-3ca3-4724-b59b-668f9b451da2-kube-api-access-fglm9\") pod \"multus-jsck4\" (UID: \"6d13a333-3ca3-4724-b59b-668f9b451da2\") " pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.524938 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.524917 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz7vr\" (UniqueName: \"kubernetes.io/projected/f772ce72-534a-48c0-88a1-de406f5eb5ca-kube-api-access-nz7vr\") pod \"ovnkube-node-vkqdz\" (UID: \"f772ce72-534a-48c0-88a1-de406f5eb5ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.527859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.527834 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p278z\" (UniqueName: \"kubernetes.io/projected/0f26d665-ebd6-4939-861f-6085d394f06b-kube-api-access-p278z\") pod \"node-resolver-lx75p\" (UID: \"0f26d665-ebd6-4939-861f-6085d394f06b\") " pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.528423 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.528392 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdqxh\" (UniqueName: \"kubernetes.io/projected/63f55018-af6a-4631-8222-0a268c900911-kube-api-access-xdqxh\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:43.529046 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.529025 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svg7t\" (UniqueName: \"kubernetes.io/projected/78d7df79-1429-47c1-9358-92675a95aa9f-kube-api-access-svg7t\") pod \"tuned-9mz5g\" (UID: \"78d7df79-1429-47c1-9358-92675a95aa9f\") " pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.530081 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.530060 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t42m9\" (UniqueName: \"kubernetes.io/projected/64eb8a18-4d89-401f-89a3-ae7dcf4d6521-kube-api-access-t42m9\") pod \"multus-additional-cni-plugins-ngkcc\" (UID: \"64eb8a18-4d89-401f-89a3-ae7dcf4d6521\") " pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.613675 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613637 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1baffed3-ce48-4637-abcd-01dd21feed17-dbus\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:43.613842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3a52f97-efe0-4757-b52b-9a1bd61d48c0-iptables-alerter-script\") pod \"iptables-alerter-hdcjp\" (UID: \"d3a52f97-efe0-4757-b52b-9a1bd61d48c0\") " pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.613842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.613842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctbmg\" (UniqueName: \"kubernetes.io/projected/2254f035-44a3-40c2-8c27-fd9f12da4c69-kube-api-access-ctbmg\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.613842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-registration-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.613842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613827 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-registration-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.613842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613826 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1baffed3-ce48-4637-abcd-01dd21feed17-dbus\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-socket-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.613901 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613921 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3a52f97-efe0-4757-b52b-9a1bd61d48c0-host-slash\") pod \"iptables-alerter-hdcjp\" (UID: \"d3a52f97-efe0-4757-b52b-9a1bd61d48c0\") " pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613829 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.613966 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret podName:1baffed3-ce48-4637-abcd-01dd21feed17 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:44.113947191 +0000 UTC m=+3.137571625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret") pod "global-pull-secret-syncer-b6g8j" (UID: "1baffed3-ce48-4637-abcd-01dd21feed17") : object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-device-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614010 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-socket-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efd21b96-4ff3-467f-89dd-f1ea27040971-host\") pod \"node-ca-zmlkq\" (UID: \"efd21b96-4ff3-467f-89dd-f1ea27040971\") " pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.613994 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3a52f97-efe0-4757-b52b-9a1bd61d48c0-host-slash\") pod \"iptables-alerter-hdcjp\" (UID: \"d3a52f97-efe0-4757-b52b-9a1bd61d48c0\") " pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614037 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-device-dir\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614060 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efd21b96-4ff3-467f-89dd-f1ea27040971-host\") pod \"node-ca-zmlkq\" (UID: \"efd21b96-4ff3-467f-89dd-f1ea27040971\") " pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/efd21b96-4ff3-467f-89dd-f1ea27040971-serviceca\") pod \"node-ca-zmlkq\" (UID: \"efd21b96-4ff3-467f-89dd-f1ea27040971\") " pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.614137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614095 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh9zg\" (UniqueName: \"kubernetes.io/projected/efd21b96-4ff3-467f-89dd-f1ea27040971-kube-api-access-mh9zg\") pod \"node-ca-zmlkq\" (UID: \"efd21b96-4ff3-467f-89dd-f1ea27040971\") " pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614281 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3a52f97-efe0-4757-b52b-9a1bd61d48c0-iptables-alerter-script\") pod \"iptables-alerter-hdcjp\" (UID: \"d3a52f97-efe0-4757-b52b-9a1bd61d48c0\") " pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614314 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk8jz\" (UniqueName: \"kubernetes.io/projected/d3a52f97-efe0-4757-b52b-9a1bd61d48c0-kube-api-access-qk8jz\") pod \"iptables-alerter-hdcjp\" (UID: \"d3a52f97-efe0-4757-b52b-9a1bd61d48c0\") " pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614332 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-sys-fs\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614348 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1baffed3-ce48-4637-abcd-01dd21feed17-kubelet-config\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614445 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/efd21b96-4ff3-467f-89dd-f1ea27040971-serviceca\") pod \"node-ca-zmlkq\" (UID: \"efd21b96-4ff3-467f-89dd-f1ea27040971\") " pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614497 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-sys-fs\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614515 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1baffed3-ce48-4637-abcd-01dd21feed17-kubelet-config\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:43.614610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.614515 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2254f035-44a3-40c2-8c27-fd9f12da4c69-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.622285 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.622261 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 15:17:43.622285 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.622287 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 15:17:43.622435 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.622301 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vr5l5 for pod openshift-network-diagnostics/network-check-target-c975x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:43.622435 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:43.622370 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5 podName:9fe3fe39-80b5-4afc-8682-54fcabb2152d nodeName:}" failed. No retries permitted until 2026-09-08 15:17:44.122352093 +0000 UTC m=+3.145976527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vr5l5" (UniqueName: "kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5") pod "network-check-target-c975x" (UID: "9fe3fe39-80b5-4afc-8682-54fcabb2152d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:43.623483 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.623459 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk8jz\" (UniqueName: \"kubernetes.io/projected/d3a52f97-efe0-4757-b52b-9a1bd61d48c0-kube-api-access-qk8jz\") pod \"iptables-alerter-hdcjp\" (UID: \"d3a52f97-efe0-4757-b52b-9a1bd61d48c0\") " pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.623603 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.623571 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctbmg\" (UniqueName: \"kubernetes.io/projected/2254f035-44a3-40c2-8c27-fd9f12da4c69-kube-api-access-ctbmg\") pod \"aws-ebs-csi-driver-node-k4sdf\" (UID: \"2254f035-44a3-40c2-8c27-fd9f12da4c69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.624405 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.624366 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh9zg\" (UniqueName: \"kubernetes.io/projected/efd21b96-4ff3-467f-89dd-f1ea27040971-kube-api-access-mh9zg\") pod \"node-ca-zmlkq\" (UID: \"efd21b96-4ff3-467f-89dd-f1ea27040971\") " pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.692686 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.692589 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:17:43.701646 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.701623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" Sep 08 15:17:43.702025 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:43.702002 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf772ce72_534a_48c0_88a1_de406f5eb5ca.slice/crio-c97ba92b99ecf68c8927defe71f8036e17e93b9d4f07ebbbf31aedf718da2544 WatchSource:0}: Error finding container c97ba92b99ecf68c8927defe71f8036e17e93b9d4f07ebbbf31aedf718da2544: Status 404 returned error can't find the container with id c97ba92b99ecf68c8927defe71f8036e17e93b9d4f07ebbbf31aedf718da2544 Sep 08 15:17:43.709342 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:43.709319 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d7df79_1429_47c1_9358_92675a95aa9f.slice/crio-403a6331c9ecf091d117f1fa32724ea82c76b39b8a5462e86fbe81e0d72fc35e WatchSource:0}: Error finding container 403a6331c9ecf091d117f1fa32724ea82c76b39b8a5462e86fbe81e0d72fc35e: Status 404 returned error can't find the container with id 403a6331c9ecf091d117f1fa32724ea82c76b39b8a5462e86fbe81e0d72fc35e Sep 08 15:17:43.712185 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.712168 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lx75p" Sep 08 15:17:43.718853 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.718835 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" Sep 08 15:17:43.720817 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:43.720794 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f26d665_ebd6_4939_861f_6085d394f06b.slice/crio-40f2b2c01316fb159f6fadab74862413e6ce141281ccd6ee3df2d2e6675a7c29 WatchSource:0}: Error finding container 40f2b2c01316fb159f6fadab74862413e6ce141281ccd6ee3df2d2e6675a7c29: Status 404 returned error can't find the container with id 40f2b2c01316fb159f6fadab74862413e6ce141281ccd6ee3df2d2e6675a7c29 Sep 08 15:17:43.726322 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.726306 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jsck4" Sep 08 15:17:43.726501 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:43.726483 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64eb8a18_4d89_401f_89a3_ae7dcf4d6521.slice/crio-103d8f3d35684e60052c6a7b13c2ca9828af89686b2a30bf16386fcac65503f3 WatchSource:0}: Error finding container 103d8f3d35684e60052c6a7b13c2ca9828af89686b2a30bf16386fcac65503f3: Status 404 returned error can't find the container with id 103d8f3d35684e60052c6a7b13c2ca9828af89686b2a30bf16386fcac65503f3 Sep 08 15:17:43.734605 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.734588 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:17:43.734731 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:43.734713 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d13a333_3ca3_4724_b59b_668f9b451da2.slice/crio-b65ac3a192cfcc00b3d9dd8471baa06e8a43b22870b1501bc6271eaa829c6cd5 WatchSource:0}: Error finding container b65ac3a192cfcc00b3d9dd8471baa06e8a43b22870b1501bc6271eaa829c6cd5: Status 404 returned error can't find the container with id b65ac3a192cfcc00b3d9dd8471baa06e8a43b22870b1501bc6271eaa829c6cd5 Sep 08 15:17:43.742180 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:43.742156 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a884a8_1be4_4cfa_bc2e_21dbcc43b79a.slice/crio-3417c3343227232447adbeeb52a73864f632136a23e37d2aa0f1f93465446195 WatchSource:0}: Error finding container 3417c3343227232447adbeeb52a73864f632136a23e37d2aa0f1f93465446195: Status 404 returned error can't find the container with id 3417c3343227232447adbeeb52a73864f632136a23e37d2aa0f1f93465446195 Sep 08 15:17:43.742748 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.742725 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hdcjp" Sep 08 15:17:43.749612 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.749587 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" Sep 08 15:17:43.750512 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:43.750490 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a52f97_efe0_4757_b52b_9a1bd61d48c0.slice/crio-476493d531ba52fdea408a5aba56eaf0f581b6568635bd37eaaca59f5b254f98 WatchSource:0}: Error finding container 476493d531ba52fdea408a5aba56eaf0f581b6568635bd37eaaca59f5b254f98: Status 404 returned error can't find the container with id 476493d531ba52fdea408a5aba56eaf0f581b6568635bd37eaaca59f5b254f98 Sep 08 15:17:43.756392 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:43.756369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zmlkq" Sep 08 15:17:43.765331 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:43.765312 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd21b96_4ff3_467f_89dd_f1ea27040971.slice/crio-466bfc0d64fceb1222c3cdc83710782e535722cc652ff76900c0a1f35bab21c1 WatchSource:0}: Error finding container 466bfc0d64fceb1222c3cdc83710782e535722cc652ff76900c0a1f35bab21c1: Status 404 returned error can't find the container with id 466bfc0d64fceb1222c3cdc83710782e535722cc652ff76900c0a1f35bab21c1 Sep 08 15:17:44.017527 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.017492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:44.017691 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:44.017640 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:44.017731 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:44.017706 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs podName:63f55018-af6a-4631-8222-0a268c900911 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:45.017689972 +0000 UTC m=+4.041314406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs") pod "network-metrics-daemon-c6lrg" (UID: "63f55018-af6a-4631-8222-0a268c900911") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:44.118335 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.118301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:44.118474 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:44.118418 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:44.118474 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:44.118471 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret podName:1baffed3-ce48-4637-abcd-01dd21feed17 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:45.118458005 +0000 UTC m=+4.142082439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret") pod "global-pull-secret-syncer-b6g8j" (UID: "1baffed3-ce48-4637-abcd-01dd21feed17") : object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:44.219251 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.219197 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:44.219431 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:44.219342 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 15:17:44.219431 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:44.219357 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 15:17:44.219431 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:44.219366 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vr5l5 for pod openshift-network-diagnostics/network-check-target-c975x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:44.219431 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:44.219424 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5 podName:9fe3fe39-80b5-4afc-8682-54fcabb2152d nodeName:}" failed. No retries permitted until 2026-09-08 15:17:45.219405231 +0000 UTC m=+4.243029685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vr5l5" (UniqueName: "kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5") pod "network-check-target-c975x" (UID: "9fe3fe39-80b5-4afc-8682-54fcabb2152d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:44.315387 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:17:44.315349 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4efd810d1fa9f921edf1aac1347d9cd.slice/crio-41f55df5d2c0a6c69a08832445ec8b5e1b1a719aa67afa6c308021b4e16c14ee WatchSource:0}: Error finding container 41f55df5d2c0a6c69a08832445ec8b5e1b1a719aa67afa6c308021b4e16c14ee: Status 404 returned error can't find the container with id 41f55df5d2c0a6c69a08832445ec8b5e1b1a719aa67afa6c308021b4e16c14ee Sep 08 15:17:44.440442 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.440404 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:12:43 +0000 UTC" deadline="2028-03-08 07:18:24.913686758 +0000 UTC" Sep 08 15:17:44.440442 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.440437 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13120h0m40.473254594s" Sep 08 15:17:44.495918 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.495880 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:44.496104 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:44.496044 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:17:44.502843 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.502765 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zmlkq" event={"ID":"efd21b96-4ff3-467f-89dd-f1ea27040971","Type":"ContainerStarted","Data":"466bfc0d64fceb1222c3cdc83710782e535722cc652ff76900c0a1f35bab21c1"} Sep 08 15:17:44.504659 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.504636 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hdcjp" event={"ID":"d3a52f97-efe0-4757-b52b-9a1bd61d48c0","Type":"ContainerStarted","Data":"476493d531ba52fdea408a5aba56eaf0f581b6568635bd37eaaca59f5b254f98"} Sep 08 15:17:44.505936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.505911 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jsck4" event={"ID":"6d13a333-3ca3-4724-b59b-668f9b451da2","Type":"ContainerStarted","Data":"b65ac3a192cfcc00b3d9dd8471baa06e8a43b22870b1501bc6271eaa829c6cd5"} Sep 08 15:17:44.507354 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.507329 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" event={"ID":"f772ce72-534a-48c0-88a1-de406f5eb5ca","Type":"ContainerStarted","Data":"c97ba92b99ecf68c8927defe71f8036e17e93b9d4f07ebbbf31aedf718da2544"} Sep 08 15:17:44.508708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.508680 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" event={"ID":"d4efd810d1fa9f921edf1aac1347d9cd","Type":"ContainerStarted","Data":"41f55df5d2c0a6c69a08832445ec8b5e1b1a719aa67afa6c308021b4e16c14ee"} Sep 08 15:17:44.509956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.509934 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" event={"ID":"2254f035-44a3-40c2-8c27-fd9f12da4c69","Type":"ContainerStarted","Data":"dd13d173d707a3d201bfca5a9687ae3ea823d44ad2260522788400ab52927f78"} Sep 08 15:17:44.511402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.511373 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vt2b6" event={"ID":"92a884a8-1be4-4cfa-bc2e-21dbcc43b79a","Type":"ContainerStarted","Data":"3417c3343227232447adbeeb52a73864f632136a23e37d2aa0f1f93465446195"} Sep 08 15:17:44.512606 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.512568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" event={"ID":"64eb8a18-4d89-401f-89a3-ae7dcf4d6521","Type":"ContainerStarted","Data":"103d8f3d35684e60052c6a7b13c2ca9828af89686b2a30bf16386fcac65503f3"} Sep 08 15:17:44.513749 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.513727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lx75p" event={"ID":"0f26d665-ebd6-4939-861f-6085d394f06b","Type":"ContainerStarted","Data":"40f2b2c01316fb159f6fadab74862413e6ce141281ccd6ee3df2d2e6675a7c29"} Sep 08 15:17:44.514916 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:44.514892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" event={"ID":"78d7df79-1429-47c1-9358-92675a95aa9f","Type":"ContainerStarted","Data":"403a6331c9ecf091d117f1fa32724ea82c76b39b8a5462e86fbe81e0d72fc35e"} Sep 08 15:17:45.025956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:45.025307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:45.025956 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.025531 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:45.025956 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.025603 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs podName:63f55018-af6a-4631-8222-0a268c900911 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:47.025577574 +0000 UTC m=+6.049202027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs") pod "network-metrics-daemon-c6lrg" (UID: "63f55018-af6a-4631-8222-0a268c900911") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:45.126396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:45.126348 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:45.126583 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.126558 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:45.126658 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.126633 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret podName:1baffed3-ce48-4637-abcd-01dd21feed17 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:47.126615338 +0000 UTC m=+6.150239777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret") pod "global-pull-secret-syncer-b6g8j" (UID: "1baffed3-ce48-4637-abcd-01dd21feed17") : object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:45.227822 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:45.227783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:45.227998 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.227978 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 15:17:45.227998 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.227998 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 15:17:45.228148 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.228010 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vr5l5 for pod openshift-network-diagnostics/network-check-target-c975x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:45.228148 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.228080 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5 podName:9fe3fe39-80b5-4afc-8682-54fcabb2152d nodeName:}" failed. No retries permitted until 2026-09-08 15:17:47.228062604 +0000 UTC m=+6.251687041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vr5l5" (UniqueName: "kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5") pod "network-check-target-c975x" (UID: "9fe3fe39-80b5-4afc-8682-54fcabb2152d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:45.440924 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:45.440852 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:12:43 +0000 UTC" deadline="2028-03-02 20:29:46.237197861 +0000 UTC" Sep 08 15:17:45.440924 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:45.440920 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12989h12m0.796281002s" Sep 08 15:17:45.495317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:45.495231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:45.495480 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.495376 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:17:45.495480 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:45.495422 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:45.495603 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:45.495569 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:17:46.495974 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:46.495941 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:46.496480 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:46.496073 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:17:47.044746 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:47.044711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:47.044930 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.044877 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:47.044994 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.044942 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs podName:63f55018-af6a-4631-8222-0a268c900911 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:51.044922716 +0000 UTC m=+10.068547164 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs") pod "network-metrics-daemon-c6lrg" (UID: "63f55018-af6a-4631-8222-0a268c900911") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:47.145771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:47.145730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:47.145952 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.145861 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:47.145952 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.145920 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret podName:1baffed3-ce48-4637-abcd-01dd21feed17 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:51.145903647 +0000 UTC m=+10.169528096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret") pod "global-pull-secret-syncer-b6g8j" (UID: "1baffed3-ce48-4637-abcd-01dd21feed17") : object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:47.247176 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:47.246512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:47.247176 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.246698 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 15:17:47.247176 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.246724 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 15:17:47.247176 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.246737 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vr5l5 for pod openshift-network-diagnostics/network-check-target-c975x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:47.247176 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.246806 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5 podName:9fe3fe39-80b5-4afc-8682-54fcabb2152d nodeName:}" failed. No retries permitted until 2026-09-08 15:17:51.246788185 +0000 UTC m=+10.270412622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vr5l5" (UniqueName: "kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5") pod "network-check-target-c975x" (UID: "9fe3fe39-80b5-4afc-8682-54fcabb2152d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:47.497506 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:47.497472 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:47.497990 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.497594 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:17:47.497990 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:47.497645 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:47.497990 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:47.497697 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:17:48.494966 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:48.494928 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:48.495191 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:48.495061 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:17:49.495805 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:49.495777 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:49.496259 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:49.495924 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:17:49.496309 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:49.496255 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:49.496347 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:49.496325 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:17:50.495620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:50.495579 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:50.495831 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:50.495726 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:17:51.080013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:51.079963 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:51.080232 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.080124 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:51.080232 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.080188 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs podName:63f55018-af6a-4631-8222-0a268c900911 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:59.080168885 +0000 UTC m=+18.103793325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs") pod "network-metrics-daemon-c6lrg" (UID: "63f55018-af6a-4631-8222-0a268c900911") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:51.180642 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:51.180604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:51.180831 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.180769 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:51.180892 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.180837 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret podName:1baffed3-ce48-4637-abcd-01dd21feed17 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:59.180817675 +0000 UTC m=+18.204442128 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret") pod "global-pull-secret-syncer-b6g8j" (UID: "1baffed3-ce48-4637-abcd-01dd21feed17") : object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:51.281764 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:51.281721 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:51.281961 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.281942 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 15:17:51.282060 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.281972 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 15:17:51.282060 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.281987 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vr5l5 for pod openshift-network-diagnostics/network-check-target-c975x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:51.282060 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.282042 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5 podName:9fe3fe39-80b5-4afc-8682-54fcabb2152d nodeName:}" failed. No retries permitted until 2026-09-08 15:17:59.282025913 +0000 UTC m=+18.305650347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vr5l5" (UniqueName: "kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5") pod "network-check-target-c975x" (UID: "9fe3fe39-80b5-4afc-8682-54fcabb2152d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:51.497203 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:51.497163 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:51.497667 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.497298 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:17:51.497734 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:51.497683 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:51.497820 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:51.497791 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:17:52.496277 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:52.495785 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:52.496277 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:52.495927 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:17:53.497710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:53.497679 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:53.498171 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:53.497793 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:17:53.498171 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:53.498083 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:53.498171 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:53.498165 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:17:54.495174 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:54.495136 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:54.495360 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:54.495261 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:17:55.495945 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:55.495909 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:55.495945 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:55.495942 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:55.496548 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:55.496060 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:17:55.496548 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:55.496190 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:17:56.495697 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:56.495673 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:56.495807 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:56.495784 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:17:57.498403 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:57.498372 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:57.498816 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:57.498376 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:57.498816 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:57.498507 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:17:57.498816 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:57.498595 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:17:58.495257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:58.495223 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:58.495453 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:58.495338 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:17:59.144207 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:59.144165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:59.144644 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.144348 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:59.144644 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.144428 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs podName:63f55018-af6a-4631-8222-0a268c900911 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:15.144405413 +0000 UTC m=+34.168029849 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs") pod "network-metrics-daemon-c6lrg" (UID: "63f55018-af6a-4631-8222-0a268c900911") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:17:59.245186 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:59.245147 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:17:59.245371 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.245313 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:59.245426 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.245391 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret podName:1baffed3-ce48-4637-abcd-01dd21feed17 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:15.245370845 +0000 UTC m=+34.268995293 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret") pod "global-pull-secret-syncer-b6g8j" (UID: "1baffed3-ce48-4637-abcd-01dd21feed17") : object "kube-system"/"original-pull-secret" not registered Sep 08 15:17:59.345838 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:59.345799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:59.346012 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.345943 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 15:17:59.346012 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.345958 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 15:17:59.346012 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.345968 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vr5l5 for pod openshift-network-diagnostics/network-check-target-c975x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:59.346162 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.346028 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5 podName:9fe3fe39-80b5-4afc-8682-54fcabb2152d nodeName:}" failed. No retries permitted until 2026-09-08 15:18:15.346009209 +0000 UTC m=+34.369633652 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vr5l5" (UniqueName: "kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5") pod "network-check-target-c975x" (UID: "9fe3fe39-80b5-4afc-8682-54fcabb2152d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:17:59.497802 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:59.497772 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:17:59.498035 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.497888 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:17:59.498035 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:17:59.497979 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:17:59.498222 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:17:59.498095 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:18:00.495165 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:00.495125 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:00.495613 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:00.495265 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:18:01.496476 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:01.496440 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:01.496912 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:01.496551 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:18:01.496912 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:01.496591 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:01.496912 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:01.496659 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:18:02.495346 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:02.495304 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:02.495520 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:02.495451 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:18:03.495106 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:03.495071 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:03.495747 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:03.495205 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:18:03.495747 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:03.495254 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:03.495747 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:03.495336 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:18:04.496302 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.495955 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:04.496903 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:04.496389 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:18:04.566414 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.566325 2578 generic.go:356] "Generic (PLEG): container finished" podID="d4efd810d1fa9f921edf1aac1347d9cd" containerID="e26c3bd823feb0266777a3a7a1067a6773117770bae48780e829c272b664487a" exitCode=0 Sep 08 15:18:04.566566 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.566413 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" event={"ID":"d4efd810d1fa9f921edf1aac1347d9cd","Type":"ContainerDied","Data":"e26c3bd823feb0266777a3a7a1067a6773117770bae48780e829c272b664487a"} Sep 08 15:18:04.568058 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.568033 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" event={"ID":"2254f035-44a3-40c2-8c27-fd9f12da4c69","Type":"ContainerStarted","Data":"eb9851d0d4834a56587859f1a0a97d165fa9c79e59412037ad2d227da638af96"} Sep 08 15:18:04.569459 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.569436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vt2b6" event={"ID":"92a884a8-1be4-4cfa-bc2e-21dbcc43b79a","Type":"ContainerStarted","Data":"2790aa50c88f35fa8d49ccdfb1542d386464fb0d9e9d633dd367607c67e93212"} Sep 08 15:18:04.571038 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.571016 2578 generic.go:356] "Generic (PLEG): container finished" podID="64eb8a18-4d89-401f-89a3-ae7dcf4d6521" containerID="95b2b8dc899238a13263f391239916f086e2eaaa91c1f2569d8ebedbcd935e4b" exitCode=0 Sep 08 15:18:04.571164 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.571082 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" event={"ID":"64eb8a18-4d89-401f-89a3-ae7dcf4d6521","Type":"ContainerDied","Data":"95b2b8dc899238a13263f391239916f086e2eaaa91c1f2569d8ebedbcd935e4b"} Sep 08 15:18:04.572584 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.572557 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lx75p" event={"ID":"0f26d665-ebd6-4939-861f-6085d394f06b","Type":"ContainerStarted","Data":"c2f75401f2262e3b0059ec37fe72ec0a35afa3e41747659271a6e3e752b29fd3"} Sep 08 15:18:04.576372 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.576345 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" event={"ID":"78d7df79-1429-47c1-9358-92675a95aa9f","Type":"ContainerStarted","Data":"04e7ec8d9b6dc02e5f337331526d1b0c61c5c29f1827a19d7c0d2b8a2bd28b28"} Sep 08 15:18:04.577774 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.577743 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal" event={"ID":"4d1459c4d11b8240fe56af12f6a1ae42","Type":"ContainerStarted","Data":"c15beaf0cad889f9aa51d1a612f612043af1f6c6dd15b0c9915c54481ec487c1"} Sep 08 15:18:04.579228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.579204 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zmlkq" event={"ID":"efd21b96-4ff3-467f-89dd-f1ea27040971","Type":"ContainerStarted","Data":"e7fa10ffb7de3af4c6753b13bbd8dce6597f45e1c2ec5db925af2ab49007ae3e"} Sep 08 15:18:04.580731 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.580710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jsck4" event={"ID":"6d13a333-3ca3-4724-b59b-668f9b451da2","Type":"ContainerStarted","Data":"5fb5d9edd89f13c69fa309b19e0575fc0b3fbd4b4b7409ef56b60811377cb0ab"} Sep 08 15:18:04.583786 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.583749 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" event={"ID":"f772ce72-534a-48c0-88a1-de406f5eb5ca","Type":"ContainerStarted","Data":"3d61aad26d5b5084ddcea02dd3b68736955a84a1252e563031eb78b022267b2b"} Sep 08 15:18:04.583884 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.583826 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" event={"ID":"f772ce72-534a-48c0-88a1-de406f5eb5ca","Type":"ContainerStarted","Data":"f095334f8dce6c36c2a066ade2605dacec59d3b8b46225c9b6d517207707bb5d"} Sep 08 15:18:04.583884 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.583876 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" event={"ID":"f772ce72-534a-48c0-88a1-de406f5eb5ca","Type":"ContainerStarted","Data":"2b582193c7662cbaa5bf5ca6724f7b9cf49eecca82bea2de294581257f34e0d7"} Sep 08 15:18:04.583977 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.583922 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" event={"ID":"f772ce72-534a-48c0-88a1-de406f5eb5ca","Type":"ContainerStarted","Data":"734151cb69be9b269f9185fee68fd112ea63c242e3c1a59b22e54700ae4daff8"} Sep 08 15:18:04.583977 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.583968 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" event={"ID":"f772ce72-534a-48c0-88a1-de406f5eb5ca","Type":"ContainerStarted","Data":"c95afef2a32562fa1b3b9c19b130890c678664561ed21268c31fbfb99c1d2b39"} Sep 08 15:18:04.584034 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.584010 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" event={"ID":"f772ce72-534a-48c0-88a1-de406f5eb5ca","Type":"ContainerStarted","Data":"b7f4e21e4c54e48db5657cc1b4d651225934a8ed110ad8a8ddd970983fc41b1a"} Sep 08 15:18:04.598339 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.598291 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zmlkq" podStartSLOduration=3.807336309 podStartE2EDuration="23.598258714s" totalImagesPullingTime="19.790922405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:04.597842755 +0000 UTC m=+23.621467214" watchObservedRunningTime="2026-09-08 15:18:04.598258714 +0000 UTC m=+23.621883171" Sep 08 15:18:04.639138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.639065 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lx75p" podStartSLOduration=3.803120003 podStartE2EDuration="23.639040508s" totalImagesPullingTime="19.835920505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:04.618021959 +0000 UTC m=+23.641646415" watchObservedRunningTime="2026-09-08 15:18:04.639040508 +0000 UTC m=+23.662664964" Sep 08 15:18:04.658102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.658044 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jsck4" podStartSLOduration=3.7760159829999997 podStartE2EDuration="23.658021276s" totalImagesPullingTime="19.882005293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:04.657440724 +0000 UTC m=+23.681065181" watchObservedRunningTime="2026-09-08 15:18:04.658021276 +0000 UTC m=+23.681645733" Sep 08 15:18:04.673875 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.673815 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vt2b6" podStartSLOduration=7.251552841 podStartE2EDuration="23.673797963s" totalImagesPullingTime="16.422245122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:04.673311755 +0000 UTC m=+23.696936211" watchObservedRunningTime="2026-09-08 15:18:04.673797963 +0000 UTC m=+23.697422454" Sep 08 15:18:04.690765 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.690715 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-25.ec2.internal" podStartSLOduration=22.690700191 podStartE2EDuration="22.690700191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:04.690565232 +0000 UTC m=+23.714189687" watchObservedRunningTime="2026-09-08 15:18:04.690700191 +0000 UTC m=+23.714324642" Sep 08 15:18:04.707548 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:04.707492 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9mz5g" podStartSLOduration=3.845795854 podStartE2EDuration="23.707478383s" totalImagesPullingTime="19.861682529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:04.707436071 +0000 UTC m=+23.731060526" watchObservedRunningTime="2026-09-08 15:18:04.707478383 +0000 UTC m=+23.731102838" Sep 08 15:18:05.495964 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:05.495929 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:05.496206 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:05.495976 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:05.496206 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:05.496076 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:18:05.496206 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:05.496191 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:18:05.587042 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:05.586998 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hdcjp" event={"ID":"d3a52f97-efe0-4757-b52b-9a1bd61d48c0","Type":"ContainerStarted","Data":"4f11fe05bb93521e27cfb7d98a5683fbd4fc1c534cbc479acaa95dca862584ea"} Sep 08 15:18:05.588953 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:05.588920 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" event={"ID":"d4efd810d1fa9f921edf1aac1347d9cd","Type":"ContainerStarted","Data":"e4e40a8ffd58d988b1447f1f7f00cc994cffcee3f4e876882c894899a51cf62f"} Sep 08 15:18:05.604737 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:05.604682 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hdcjp" podStartSLOduration=4.784953447 podStartE2EDuration="24.604662512s" totalImagesPullingTime="19.819709065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:05.604323147 +0000 UTC m=+24.627947606" watchObservedRunningTime="2026-09-08 15:18:05.604662512 +0000 UTC m=+24.628286968" Sep 08 15:18:05.621602 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:05.621547 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-25.ec2.internal" podStartSLOduration=23.621530702 podStartE2EDuration="23.621530702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:05.621177276 +0000 UTC m=+24.644801732" watchObservedRunningTime="2026-09-08 15:18:05.621530702 +0000 UTC m=+24.645155157" Sep 08 15:18:05.676642 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:05.676604 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:18:05.677546 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:05.677522 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:18:06.401336 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:06.401139 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 08 15:18:06.473513 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:06.473400 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T15:18:06.401324045Z","UUID":"8fcbf8ab-91e4-4606-a843-0751b9e9fe2f","Handler":null,"Name":"","Endpoint":""} Sep 08 15:18:06.475172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:06.475149 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 08 15:18:06.475298 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:06.475181 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 08 15:18:06.495542 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:06.495504 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:06.495693 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:06.495630 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:18:06.595182 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:06.595046 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" event={"ID":"f772ce72-534a-48c0-88a1-de406f5eb5ca","Type":"ContainerStarted","Data":"3208bafc72cd805b2abdf613f2c20a768335a6050a5e3a1b3b31267fa3dd0e3c"} Sep 08 15:18:06.596973 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:06.596944 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" event={"ID":"2254f035-44a3-40c2-8c27-fd9f12da4c69","Type":"ContainerStarted","Data":"914831128f1f31f694b550c081cec2e13dcd68c51048400d183c831acc9a1184"} Sep 08 15:18:06.597324 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:06.597294 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:18:06.597785 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:06.597767 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vt2b6" Sep 08 15:18:07.495777 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:07.495739 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:07.495956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:07.495786 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:07.495956 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:07.495885 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:18:07.496050 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:07.496024 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:18:08.495790 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:08.495758 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:08.496263 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:08.495890 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:18:09.495257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:09.495216 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:09.495257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:09.495252 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:09.495475 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:09.495378 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:18:09.495531 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:09.495496 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:18:09.608349 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:09.608307 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" event={"ID":"f772ce72-534a-48c0-88a1-de406f5eb5ca","Type":"ContainerStarted","Data":"96ae5e92fc86abd6769b5dad7898ee25db01b0e24dac1bfd9274c0efff689dcb"} Sep 08 15:18:09.608753 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:09.608628 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:18:09.627481 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:09.627451 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:18:09.638953 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:09.638898 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" podStartSLOduration=8.714044389 podStartE2EDuration="28.638884064s" totalImagesPullingTime="19.924839675s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:09.637940061 +0000 UTC m=+28.661564517" watchObservedRunningTime="2026-09-08 15:18:09.638884064 +0000 UTC m=+28.662508540" Sep 08 15:18:10.496950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.496776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:10.497141 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:10.497050 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:18:10.555271 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.555232 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b6g8j"] Sep 08 15:18:10.558442 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.558415 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c975x"] Sep 08 15:18:10.558573 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.558542 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:10.558726 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:10.558701 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:18:10.559198 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.559168 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c6lrg"] Sep 08 15:18:10.559298 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.559275 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:10.559405 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:10.559382 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:18:10.610243 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.610165 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:10.610995 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:10.610259 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:18:10.610995 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.610345 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:18:10.610995 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.610843 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:18:10.626025 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:10.625994 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:18:11.613866 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:11.613826 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" event={"ID":"2254f035-44a3-40c2-8c27-fd9f12da4c69","Type":"ContainerStarted","Data":"46e2781052065c2c542f9d4ca6900b6b2e8796ce71bddb66f914ad99f15946c1"} Sep 08 15:18:11.615653 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:11.615628 2578 generic.go:356] "Generic (PLEG): container finished" podID="64eb8a18-4d89-401f-89a3-ae7dcf4d6521" containerID="93c2dd3cad83ceb7183da7e61b6159a89c502368ab450faf90d70124045d7e7d" exitCode=0 Sep 08 15:18:11.615741 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:11.615719 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" event={"ID":"64eb8a18-4d89-401f-89a3-ae7dcf4d6521","Type":"ContainerDied","Data":"93c2dd3cad83ceb7183da7e61b6159a89c502368ab450faf90d70124045d7e7d"} Sep 08 15:18:11.635468 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:11.635416 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4sdf" podStartSLOduration=4.3028574729999995 podStartE2EDuration="30.635399572s" totalImagesPullingTime="26.332542099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:11.635369857 +0000 UTC m=+30.658994313" watchObservedRunningTime="2026-09-08 15:18:11.635399572 +0000 UTC m=+30.659024011" Sep 08 15:18:12.495079 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:12.495042 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:12.495261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:12.495042 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:12.495261 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:12.495163 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:18:12.495261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:12.495047 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:12.495354 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:12.495302 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:18:12.495408 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:12.495389 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:18:13.622234 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:13.621977 2578 generic.go:356] "Generic (PLEG): container finished" podID="64eb8a18-4d89-401f-89a3-ae7dcf4d6521" containerID="44a1e7bbd1cacf903a912f663078d4faafdd8d44df5c1c4e27b8bac1f9c71b3d" exitCode=0 Sep 08 15:18:13.622234 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:13.622066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" event={"ID":"64eb8a18-4d89-401f-89a3-ae7dcf4d6521","Type":"ContainerDied","Data":"44a1e7bbd1cacf903a912f663078d4faafdd8d44df5c1c4e27b8bac1f9c71b3d"} Sep 08 15:18:14.495922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:14.495885 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:14.496099 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:14.495987 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:14.496099 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:14.496081 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6g8j" podUID="1baffed3-ce48-4637-abcd-01dd21feed17" Sep 08 15:18:14.496196 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:14.495990 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c975x" podUID="9fe3fe39-80b5-4afc-8682-54fcabb2152d" Sep 08 15:18:14.496196 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:14.495993 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:14.496251 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:14.496195 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6lrg" podUID="63f55018-af6a-4631-8222-0a268c900911" Sep 08 15:18:14.628260 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:14.628227 2578 generic.go:356] "Generic (PLEG): container finished" podID="64eb8a18-4d89-401f-89a3-ae7dcf4d6521" containerID="b28baf3cea7c6b2b05b99d3631a01467d7c852d9402514f3caed6a9341d5b838" exitCode=0 Sep 08 15:18:14.628603 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:14.628273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" event={"ID":"64eb8a18-4d89-401f-89a3-ae7dcf4d6521","Type":"ContainerDied","Data":"b28baf3cea7c6b2b05b99d3631a01467d7c852d9402514f3caed6a9341d5b838"} Sep 08 15:18:14.820297 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:14.820268 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lx75p_0f26d665-ebd6-4939-861f-6085d394f06b/dns-node-resolver/0.log" Sep 08 15:18:15.161653 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:15.161612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:15.161829 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:15.161744 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:18:15.161829 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:15.161818 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs podName:63f55018-af6a-4631-8222-0a268c900911 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:47.161798086 +0000 UTC m=+66.185422533 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs") pod "network-metrics-daemon-c6lrg" (UID: "63f55018-af6a-4631-8222-0a268c900911") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 15:18:15.262520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:15.262483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:15.262746 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:15.262639 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 15:18:15.262746 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:15.262716 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret podName:1baffed3-ce48-4637-abcd-01dd21feed17 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:47.262699427 +0000 UTC m=+66.286323865 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret") pod "global-pull-secret-syncer-b6g8j" (UID: "1baffed3-ce48-4637-abcd-01dd21feed17") : object "kube-system"/"original-pull-secret" not registered Sep 08 15:18:15.363348 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:15.363308 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:15.363571 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:15.363498 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 15:18:15.363571 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:15.363522 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 15:18:15.363571 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:15.363532 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vr5l5 for pod openshift-network-diagnostics/network-check-target-c975x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:18:15.363673 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:15.363585 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5 podName:9fe3fe39-80b5-4afc-8682-54fcabb2152d nodeName:}" failed. No retries permitted until 2026-09-08 15:18:47.363571466 +0000 UTC m=+66.387195901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vr5l5" (UniqueName: "kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5") pod "network-check-target-c975x" (UID: "9fe3fe39-80b5-4afc-8682-54fcabb2152d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 15:18:15.785204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:15.785175 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zmlkq_efd21b96-4ff3-467f-89dd-f1ea27040971/node-ca/0.log" Sep 08 15:18:16.289204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.289175 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-25.ec2.internal" event="NodeReady" Sep 08 15:18:16.289381 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.289340 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 08 15:18:16.495168 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.495127 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:16.495168 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.495169 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:16.495421 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.495127 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:16.498126 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.498089 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np4gb\"" Sep 08 15:18:16.498269 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.498090 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 15:18:16.498269 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.498096 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 15:18:16.498269 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.498204 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 15:18:16.498432 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.498374 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 15:18:16.498604 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:16.498587 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76z9f\"" Sep 08 15:18:19.643542 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:19.643358 2578 generic.go:356] "Generic (PLEG): container finished" podID="64eb8a18-4d89-401f-89a3-ae7dcf4d6521" containerID="8b35dc773176eeccb69e3aec045b6c3ba2cd95e25491b91b7227d4b6f4ab3c3f" exitCode=0 Sep 08 15:18:19.643542 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:19.643437 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" event={"ID":"64eb8a18-4d89-401f-89a3-ae7dcf4d6521","Type":"ContainerDied","Data":"8b35dc773176eeccb69e3aec045b6c3ba2cd95e25491b91b7227d4b6f4ab3c3f"} Sep 08 15:18:20.649582 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:20.649549 2578 generic.go:356] "Generic (PLEG): container finished" podID="64eb8a18-4d89-401f-89a3-ae7dcf4d6521" containerID="f2571adb240dcf568ee615ade05b18867227abec84bb6bf96530d42c9813b4f2" exitCode=0 Sep 08 15:18:20.649950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:20.649600 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" event={"ID":"64eb8a18-4d89-401f-89a3-ae7dcf4d6521","Type":"ContainerDied","Data":"f2571adb240dcf568ee615ade05b18867227abec84bb6bf96530d42c9813b4f2"} Sep 08 15:18:21.655503 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:21.655464 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" event={"ID":"64eb8a18-4d89-401f-89a3-ae7dcf4d6521","Type":"ContainerStarted","Data":"12b403b0294f979514c448a0af2bca3f2a383a1936e86236a17150e55dd86a8f"} Sep 08 15:18:21.687982 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:21.687931 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ngkcc" podStartSLOduration=11.198997622 podStartE2EDuration="40.687913803s" totalImagesPullingTime="29.488916181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:21.687500094 +0000 UTC m=+40.711124551" watchObservedRunningTime="2026-09-08 15:18:21.687913803 +0000 UTC m=+40.711538259" Sep 08 15:18:35.992718 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:35.992688 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wsp27"] Sep 08 15:18:35.994738 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:35.994721 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:35.998384 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:35.998354 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wln7f\"" Sep 08 15:18:35.998590 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:35.998572 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 08 15:18:35.998796 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:35.998781 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 08 15:18:35.999505 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:35.999493 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 08 15:18:35.999583 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:35.999566 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 08 15:18:36.015039 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.015009 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wsp27"] Sep 08 15:18:36.035133 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.035083 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nb2xs"] Sep 08 15:18:36.036861 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.036845 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nb2xs" Sep 08 15:18:36.039491 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.039458 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 08 15:18:36.039491 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.039469 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9zz4p\"" Sep 08 15:18:36.039491 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.039469 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 08 15:18:36.040758 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.040740 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 08 15:18:36.042004 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.041925 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-crrxx"] Sep 08 15:18:36.043910 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.043888 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.044821 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.044801 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6866fd6c9c-fbltn"] Sep 08 15:18:36.046465 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.046438 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.046755 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.046731 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 08 15:18:36.046939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.046925 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-65z9v\"" Sep 08 15:18:36.047033 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.047011 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 08 15:18:36.049574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.049557 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 08 15:18:36.049832 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.049816 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 08 15:18:36.050062 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.050044 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 08 15:18:36.050179 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.050047 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-smd6z\"" Sep 08 15:18:36.052321 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.052300 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nb2xs"] Sep 08 15:18:36.057940 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.057892 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-crrxx"] Sep 08 15:18:36.060830 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.060804 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 08 15:18:36.075999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.075959 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6866fd6c9c-fbltn"] Sep 08 15:18:36.105775 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.105739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/204e0ebc-84af-4455-84d7-fb5fafbdf069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.105775 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.105778 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc28m\" (UniqueName: \"kubernetes.io/projected/204e0ebc-84af-4455-84d7-fb5fafbdf069-kube-api-access-nc28m\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.105980 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.105842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/204e0ebc-84af-4455-84d7-fb5fafbdf069-crio-socket\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.105980 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.105915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/204e0ebc-84af-4455-84d7-fb5fafbdf069-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.105980 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.105947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/204e0ebc-84af-4455-84d7-fb5fafbdf069-data-volume\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.207166 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207093 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/204e0ebc-84af-4455-84d7-fb5fafbdf069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.207166 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc28m\" (UniqueName: \"kubernetes.io/projected/204e0ebc-84af-4455-84d7-fb5fafbdf069-kube-api-access-nc28m\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.207419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207194 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w55bs\" (UniqueName: \"kubernetes.io/projected/ab614993-0c65-45df-9703-74ad6c70a1a4-kube-api-access-w55bs\") pod \"ingress-canary-nb2xs\" (UID: \"ab614993-0c65-45df-9703-74ad6c70a1a4\") " pod="openshift-ingress-canary/ingress-canary-nb2xs" Sep 08 15:18:36.207419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p2dq\" (UniqueName: \"kubernetes.io/projected/1b72473a-b0b1-4b67-913f-930bfffbbda0-kube-api-access-4p2dq\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.207419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207238 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/204e0ebc-84af-4455-84d7-fb5fafbdf069-crio-socket\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.207419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207257 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b72473a-b0b1-4b67-913f-930bfffbbda0-registry-certificates\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.207419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4ef42-8f74-405d-ba49-bef1f7c40c82-config-volume\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.207419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkw76\" (UniqueName: \"kubernetes.io/projected/65e4ef42-8f74-405d-ba49-bef1f7c40c82-kube-api-access-gkw76\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.207419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207336 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/204e0ebc-84af-4455-84d7-fb5fafbdf069-crio-socket\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.207419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b72473a-b0b1-4b67-913f-930bfffbbda0-image-registry-private-configuration\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207444 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b72473a-b0b1-4b67-913f-930bfffbbda0-ca-trust-extracted\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b72473a-b0b1-4b67-913f-930bfffbbda0-trusted-ca\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b72473a-b0b1-4b67-913f-930bfffbbda0-installation-pull-secrets\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/204e0ebc-84af-4455-84d7-fb5fafbdf069-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b72473a-b0b1-4b67-913f-930bfffbbda0-registry-tls\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/204e0ebc-84af-4455-84d7-fb5fafbdf069-data-volume\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab614993-0c65-45df-9703-74ad6c70a1a4-cert\") pod \"ingress-canary-nb2xs\" (UID: \"ab614993-0c65-45df-9703-74ad6c70a1a4\") " pod="openshift-ingress-canary/ingress-canary-nb2xs" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e4ef42-8f74-405d-ba49-bef1f7c40c82-metrics-tls\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207691 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65e4ef42-8f74-405d-ba49-bef1f7c40c82-tmp-dir\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.207771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207705 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b72473a-b0b1-4b67-913f-930bfffbbda0-bound-sa-token\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.208403 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.207920 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/204e0ebc-84af-4455-84d7-fb5fafbdf069-data-volume\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.208403 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.208181 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/204e0ebc-84af-4455-84d7-fb5fafbdf069-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.211339 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.211323 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/204e0ebc-84af-4455-84d7-fb5fafbdf069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.218441 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.218417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc28m\" (UniqueName: \"kubernetes.io/projected/204e0ebc-84af-4455-84d7-fb5fafbdf069-kube-api-access-nc28m\") pod \"insights-runtime-extractor-wsp27\" (UID: \"204e0ebc-84af-4455-84d7-fb5fafbdf069\") " pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.303339 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.303242 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wsp27" Sep 08 15:18:36.308568 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b72473a-b0b1-4b67-913f-930bfffbbda0-registry-tls\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.308639 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308592 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab614993-0c65-45df-9703-74ad6c70a1a4-cert\") pod \"ingress-canary-nb2xs\" (UID: \"ab614993-0c65-45df-9703-74ad6c70a1a4\") " pod="openshift-ingress-canary/ingress-canary-nb2xs" Sep 08 15:18:36.308639 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e4ef42-8f74-405d-ba49-bef1f7c40c82-metrics-tls\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.308639 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65e4ef42-8f74-405d-ba49-bef1f7c40c82-tmp-dir\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.308777 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b72473a-b0b1-4b67-913f-930bfffbbda0-bound-sa-token\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.308777 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308702 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55bs\" (UniqueName: \"kubernetes.io/projected/ab614993-0c65-45df-9703-74ad6c70a1a4-kube-api-access-w55bs\") pod \"ingress-canary-nb2xs\" (UID: \"ab614993-0c65-45df-9703-74ad6c70a1a4\") " pod="openshift-ingress-canary/ingress-canary-nb2xs" Sep 08 15:18:36.308777 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p2dq\" (UniqueName: \"kubernetes.io/projected/1b72473a-b0b1-4b67-913f-930bfffbbda0-kube-api-access-4p2dq\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.308777 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b72473a-b0b1-4b67-913f-930bfffbbda0-registry-certificates\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.308966 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4ef42-8f74-405d-ba49-bef1f7c40c82-config-volume\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.308966 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkw76\" (UniqueName: \"kubernetes.io/projected/65e4ef42-8f74-405d-ba49-bef1f7c40c82-kube-api-access-gkw76\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.308966 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b72473a-b0b1-4b67-913f-930bfffbbda0-image-registry-private-configuration\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.308966 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b72473a-b0b1-4b67-913f-930bfffbbda0-ca-trust-extracted\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.308966 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308897 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b72473a-b0b1-4b67-913f-930bfffbbda0-trusted-ca\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.308966 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308921 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b72473a-b0b1-4b67-913f-930bfffbbda0-installation-pull-secrets\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.309281 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.308994 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65e4ef42-8f74-405d-ba49-bef1f7c40c82-tmp-dir\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.309694 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.309423 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b72473a-b0b1-4b67-913f-930bfffbbda0-ca-trust-extracted\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.309694 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.309438 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4ef42-8f74-405d-ba49-bef1f7c40c82-config-volume\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.310245 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.310220 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b72473a-b0b1-4b67-913f-930bfffbbda0-registry-certificates\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.310415 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.310363 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b72473a-b0b1-4b67-913f-930bfffbbda0-trusted-ca\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.312196 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.311943 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab614993-0c65-45df-9703-74ad6c70a1a4-cert\") pod \"ingress-canary-nb2xs\" (UID: \"ab614993-0c65-45df-9703-74ad6c70a1a4\") " pod="openshift-ingress-canary/ingress-canary-nb2xs" Sep 08 15:18:36.312196 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.311989 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e4ef42-8f74-405d-ba49-bef1f7c40c82-metrics-tls\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.312196 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.312044 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b72473a-b0b1-4b67-913f-930bfffbbda0-installation-pull-secrets\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.312372 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.312260 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b72473a-b0b1-4b67-913f-930bfffbbda0-registry-tls\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.312721 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.312699 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b72473a-b0b1-4b67-913f-930bfffbbda0-image-registry-private-configuration\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.321178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.321142 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p2dq\" (UniqueName: \"kubernetes.io/projected/1b72473a-b0b1-4b67-913f-930bfffbbda0-kube-api-access-4p2dq\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.322044 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.322014 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b72473a-b0b1-4b67-913f-930bfffbbda0-bound-sa-token\") pod \"image-registry-6866fd6c9c-fbltn\" (UID: \"1b72473a-b0b1-4b67-913f-930bfffbbda0\") " pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.322305 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.322286 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkw76\" (UniqueName: \"kubernetes.io/projected/65e4ef42-8f74-405d-ba49-bef1f7c40c82-kube-api-access-gkw76\") pod \"dns-default-crrxx\" (UID: \"65e4ef42-8f74-405d-ba49-bef1f7c40c82\") " pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.322604 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.322578 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w55bs\" (UniqueName: \"kubernetes.io/projected/ab614993-0c65-45df-9703-74ad6c70a1a4-kube-api-access-w55bs\") pod \"ingress-canary-nb2xs\" (UID: \"ab614993-0c65-45df-9703-74ad6c70a1a4\") " pod="openshift-ingress-canary/ingress-canary-nb2xs" Sep 08 15:18:36.346945 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.346905 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nb2xs" Sep 08 15:18:36.355242 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.355221 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:36.365729 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.365467 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:36.593926 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.593840 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wsp27"] Sep 08 15:18:36.596910 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:36.596879 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod204e0ebc_84af_4455_84d7_fb5fafbdf069.slice/crio-05821e88f9a037157df4082c04e134c288288946d770e49da3144ffe63a38198 WatchSource:0}: Error finding container 05821e88f9a037157df4082c04e134c288288946d770e49da3144ffe63a38198: Status 404 returned error can't find the container with id 05821e88f9a037157df4082c04e134c288288946d770e49da3144ffe63a38198 Sep 08 15:18:36.641047 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.641018 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nb2xs"] Sep 08 15:18:36.643858 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.643832 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-crrxx"] Sep 08 15:18:36.645620 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:36.645592 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab614993_0c65_45df_9703_74ad6c70a1a4.slice/crio-41e9aaf69528e020410f91db55297d4dba4846fba8728e9ba8c06c440e0cc561 WatchSource:0}: Error finding container 41e9aaf69528e020410f91db55297d4dba4846fba8728e9ba8c06c440e0cc561: Status 404 returned error can't find the container with id 41e9aaf69528e020410f91db55297d4dba4846fba8728e9ba8c06c440e0cc561 Sep 08 15:18:36.646375 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:36.646348 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e4ef42_8f74_405d_ba49_bef1f7c40c82.slice/crio-25ff88bb47c0ebc90fb6309dc3cfca10b5629ac88453cf6a4cd4586e64b37cc1 WatchSource:0}: Error finding container 25ff88bb47c0ebc90fb6309dc3cfca10b5629ac88453cf6a4cd4586e64b37cc1: Status 404 returned error can't find the container with id 25ff88bb47c0ebc90fb6309dc3cfca10b5629ac88453cf6a4cd4586e64b37cc1 Sep 08 15:18:36.663977 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.663948 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6866fd6c9c-fbltn"] Sep 08 15:18:36.666579 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:36.666556 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b72473a_b0b1_4b67_913f_930bfffbbda0.slice/crio-4fd06059b3166ab5c62413c3e9c6a38ecc73222c75a1bbab19518f6e87062975 WatchSource:0}: Error finding container 4fd06059b3166ab5c62413c3e9c6a38ecc73222c75a1bbab19518f6e87062975: Status 404 returned error can't find the container with id 4fd06059b3166ab5c62413c3e9c6a38ecc73222c75a1bbab19518f6e87062975 Sep 08 15:18:36.691022 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.690993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-crrxx" event={"ID":"65e4ef42-8f74-405d-ba49-bef1f7c40c82","Type":"ContainerStarted","Data":"25ff88bb47c0ebc90fb6309dc3cfca10b5629ac88453cf6a4cd4586e64b37cc1"} Sep 08 15:18:36.692598 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.692576 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsp27" event={"ID":"204e0ebc-84af-4455-84d7-fb5fafbdf069","Type":"ContainerStarted","Data":"8eaa6fadb004a64e6226f4c8525182ef259da29e2331e63f51228bd508e714b5"} Sep 08 15:18:36.692698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.692632 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsp27" event={"ID":"204e0ebc-84af-4455-84d7-fb5fafbdf069","Type":"ContainerStarted","Data":"05821e88f9a037157df4082c04e134c288288946d770e49da3144ffe63a38198"} Sep 08 15:18:36.693633 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.693605 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" event={"ID":"1b72473a-b0b1-4b67-913f-930bfffbbda0","Type":"ContainerStarted","Data":"4fd06059b3166ab5c62413c3e9c6a38ecc73222c75a1bbab19518f6e87062975"} Sep 08 15:18:36.694707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:36.694662 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nb2xs" event={"ID":"ab614993-0c65-45df-9703-74ad6c70a1a4","Type":"ContainerStarted","Data":"41e9aaf69528e020410f91db55297d4dba4846fba8728e9ba8c06c440e0cc561"} Sep 08 15:18:37.038707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.038645 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5"] Sep 08 15:18:37.042762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.042739 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" Sep 08 15:18:37.045910 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.045885 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-b6hd6\"" Sep 08 15:18:37.046439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.046419 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 08 15:18:37.057124 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.057077 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5"] Sep 08 15:18:37.214852 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.214811 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b97f9898-ae7b-4bb1-b7b7-7a8b2cea6aaf-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-28xk5\" (UID: \"b97f9898-ae7b-4bb1-b7b7-7a8b2cea6aaf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" Sep 08 15:18:37.315518 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.315423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b97f9898-ae7b-4bb1-b7b7-7a8b2cea6aaf-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-28xk5\" (UID: \"b97f9898-ae7b-4bb1-b7b7-7a8b2cea6aaf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" Sep 08 15:18:37.318252 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.318227 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b97f9898-ae7b-4bb1-b7b7-7a8b2cea6aaf-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-28xk5\" (UID: \"b97f9898-ae7b-4bb1-b7b7-7a8b2cea6aaf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" Sep 08 15:18:37.351400 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.351367 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" Sep 08 15:18:37.629755 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.629669 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5"] Sep 08 15:18:37.633669 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:37.633643 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb97f9898_ae7b_4bb1_b7b7_7a8b2cea6aaf.slice/crio-369683243eeda4676f50acc82deaf7a7913e0623a797e92db1f79362643fc030 WatchSource:0}: Error finding container 369683243eeda4676f50acc82deaf7a7913e0623a797e92db1f79362643fc030: Status 404 returned error can't find the container with id 369683243eeda4676f50acc82deaf7a7913e0623a797e92db1f79362643fc030 Sep 08 15:18:37.699581 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.699049 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" event={"ID":"1b72473a-b0b1-4b67-913f-930bfffbbda0","Type":"ContainerStarted","Data":"ade0b62a0aab44646474c38af56c378a461bc1328677e5fbd46caa83f6d93869"} Sep 08 15:18:37.699581 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.699172 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:37.700239 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.700206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" event={"ID":"b97f9898-ae7b-4bb1-b7b7-7a8b2cea6aaf","Type":"ContainerStarted","Data":"369683243eeda4676f50acc82deaf7a7913e0623a797e92db1f79362643fc030"} Sep 08 15:18:37.723910 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:37.722467 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" podStartSLOduration=1.722437417 podStartE2EDuration="1.722437417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:37.721366074 +0000 UTC m=+56.744990532" watchObservedRunningTime="2026-09-08 15:18:37.722437417 +0000 UTC m=+56.746061874" Sep 08 15:18:38.704753 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:38.704709 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsp27" event={"ID":"204e0ebc-84af-4455-84d7-fb5fafbdf069","Type":"ContainerStarted","Data":"e5bb137837bf6e9d22e44658ed2eb04488798b428d667ae3275c58f29bedc723"} Sep 08 15:18:41.717937 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.717892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-crrxx" event={"ID":"65e4ef42-8f74-405d-ba49-bef1f7c40c82","Type":"ContainerStarted","Data":"037ceff4656eb770d721db2870374ff2904d96dc6295ca3399b5930a9cc376ad"} Sep 08 15:18:41.718489 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.717959 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-crrxx" event={"ID":"65e4ef42-8f74-405d-ba49-bef1f7c40c82","Type":"ContainerStarted","Data":"073a84894d70ea3419f4639b8ad998784f424439b40f10d65e717d261ac15597"} Sep 08 15:18:41.718489 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.718038 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:41.719490 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.719464 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" event={"ID":"b97f9898-ae7b-4bb1-b7b7-7a8b2cea6aaf","Type":"ContainerStarted","Data":"53db7fd12b14b7a1df308148c4c7913f31e6e553bd95d79f82a520478958600f"} Sep 08 15:18:41.719649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.719629 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" Sep 08 15:18:41.721798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.721761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsp27" event={"ID":"204e0ebc-84af-4455-84d7-fb5fafbdf069","Type":"ContainerStarted","Data":"990a152d3adf1148ad80ce897214f15db8d9d636eca82812e7eba014e6bc120d"} Sep 08 15:18:41.723644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.723624 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nb2xs" event={"ID":"ab614993-0c65-45df-9703-74ad6c70a1a4","Type":"ContainerStarted","Data":"4eba9e162b97f4f8c9e83c7d35fd286f591b918fdef6730b48d76eabf5b1538c"} Sep 08 15:18:41.724819 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.724800 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" Sep 08 15:18:41.737684 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.737642 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-crrxx" podStartSLOduration=2.047352619 podStartE2EDuration="5.737628715s" totalImagesPullingTime="3.690276096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:41.736667487 +0000 UTC m=+60.760291946" watchObservedRunningTime="2026-09-08 15:18:41.737628715 +0000 UTC m=+60.761253170" Sep 08 15:18:41.754257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.754210 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-28xk5" podStartSLOduration=2.05648213 podStartE2EDuration="4.754193961s" totalImagesPullingTime="2.697711831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:41.753362962 +0000 UTC m=+60.776987417" watchObservedRunningTime="2026-09-08 15:18:41.754193961 +0000 UTC m=+60.777818417" Sep 08 15:18:41.770035 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.769954 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nb2xs" podStartSLOduration=2.083902448 podStartE2EDuration="5.769933524s" totalImagesPullingTime="3.686031076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:41.769453871 +0000 UTC m=+60.793078327" watchObservedRunningTime="2026-09-08 15:18:41.769933524 +0000 UTC m=+60.793557978" Sep 08 15:18:41.792567 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:41.792521 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wsp27" podStartSLOduration=2.425983635 podStartE2EDuration="6.792502185s" totalImagesPullingTime="4.36651855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:18:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:41.791700934 +0000 UTC m=+60.815325391" watchObservedRunningTime="2026-09-08 15:18:41.792502185 +0000 UTC m=+60.816126640" Sep 08 15:18:42.116903 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.116814 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-jrdsd"] Sep 08 15:18:42.120845 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.120829 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.123585 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.123558 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 08 15:18:42.123720 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.123662 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 08 15:18:42.124281 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.124251 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 08 15:18:42.125042 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.124998 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 08 15:18:42.125212 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.125193 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7dm9j\"" Sep 08 15:18:42.125347 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.125331 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 08 15:18:42.130086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.130059 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-jrdsd"] Sep 08 15:18:42.259950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.259914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34279f8e-45c5-4571-8055-baddd408eb55-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.259950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.259952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34279f8e-45c5-4571-8055-baddd408eb55-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.260213 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.259982 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34279f8e-45c5-4571-8055-baddd408eb55-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.260213 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.260052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzwhf\" (UniqueName: \"kubernetes.io/projected/34279f8e-45c5-4571-8055-baddd408eb55-kube-api-access-xzwhf\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.361416 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.361378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34279f8e-45c5-4571-8055-baddd408eb55-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.361416 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.361420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34279f8e-45c5-4571-8055-baddd408eb55-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.361579 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.361550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34279f8e-45c5-4571-8055-baddd408eb55-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.361631 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.361616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzwhf\" (UniqueName: \"kubernetes.io/projected/34279f8e-45c5-4571-8055-baddd408eb55-kube-api-access-xzwhf\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.362170 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.362145 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34279f8e-45c5-4571-8055-baddd408eb55-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.364022 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.363999 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34279f8e-45c5-4571-8055-baddd408eb55-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.364242 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.364224 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34279f8e-45c5-4571-8055-baddd408eb55-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.370368 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.370321 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzwhf\" (UniqueName: \"kubernetes.io/projected/34279f8e-45c5-4571-8055-baddd408eb55-kube-api-access-xzwhf\") pod \"prometheus-operator-6f4946db56-jrdsd\" (UID: \"34279f8e-45c5-4571-8055-baddd408eb55\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.432237 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.432195 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" Sep 08 15:18:42.629288 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.629203 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkqdz" Sep 08 15:18:42.703979 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.703945 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-jrdsd"] Sep 08 15:18:42.707262 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:42.707234 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34279f8e_45c5_4571_8055_baddd408eb55.slice/crio-c131eec0200c611ed5d94f59ad525c26d3baf907f23dae33581d6ef65392d853 WatchSource:0}: Error finding container c131eec0200c611ed5d94f59ad525c26d3baf907f23dae33581d6ef65392d853: Status 404 returned error can't find the container with id c131eec0200c611ed5d94f59ad525c26d3baf907f23dae33581d6ef65392d853 Sep 08 15:18:42.727641 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:42.727599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" event={"ID":"34279f8e-45c5-4571-8055-baddd408eb55","Type":"ContainerStarted","Data":"c131eec0200c611ed5d94f59ad525c26d3baf907f23dae33581d6ef65392d853"} Sep 08 15:18:44.735746 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:44.735715 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" event={"ID":"34279f8e-45c5-4571-8055-baddd408eb55","Type":"ContainerStarted","Data":"105297a62cf97e88fc9c2d9d5336c6861ab1686b775e3c8ee0dc87dafc87ea10"} Sep 08 15:18:45.743576 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:45.743539 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" event={"ID":"34279f8e-45c5-4571-8055-baddd408eb55","Type":"ContainerStarted","Data":"3af8d9906881b2a3926a7839db99a261d040e5afd298b7951d062d25e1049209"} Sep 08 15:18:45.765274 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:45.765220 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-jrdsd" podStartSLOduration=1.8454953710000002 podStartE2EDuration="3.76520431s" totalImagesPullingTime="1.919708939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:45.76480033 +0000 UTC m=+64.788424786" watchObservedRunningTime="2026-09-08 15:18:45.76520431 +0000 UTC m=+64.788828763" Sep 08 15:18:47.199735 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.199697 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:47.202658 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.202638 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 15:18:47.213386 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.213352 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63f55018-af6a-4631-8222-0a268c900911-metrics-certs\") pod \"network-metrics-daemon-c6lrg\" (UID: \"63f55018-af6a-4631-8222-0a268c900911\") " pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:47.301074 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.301033 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:47.303798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.303778 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 15:18:47.314594 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.314564 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1baffed3-ce48-4637-abcd-01dd21feed17-original-pull-secret\") pod \"global-pull-secret-syncer-b6g8j\" (UID: \"1baffed3-ce48-4637-abcd-01dd21feed17\") " pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:47.402212 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.402158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:47.405348 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.405330 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 15:18:47.407947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.407934 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76z9f\"" Sep 08 15:18:47.411135 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.411103 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6g8j" Sep 08 15:18:47.415430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.415410 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6lrg" Sep 08 15:18:47.415995 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.415979 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 15:18:47.426586 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.426424 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr5l5\" (UniqueName: \"kubernetes.io/projected/9fe3fe39-80b5-4afc-8682-54fcabb2152d-kube-api-access-vr5l5\") pod \"network-check-target-c975x\" (UID: \"9fe3fe39-80b5-4afc-8682-54fcabb2152d\") " pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:47.537792 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.537755 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72"] Sep 08 15:18:47.542985 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.542965 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.545505 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.545467 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-stxmz"] Sep 08 15:18:47.545913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.545885 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 08 15:18:47.546045 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.545900 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qhms7\"" Sep 08 15:18:47.546045 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.545900 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 08 15:18:47.549203 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.549184 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.551885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.551864 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 08 15:18:47.552022 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.551929 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 08 15:18:47.552022 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.551930 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-f9vj6\"" Sep 08 15:18:47.552022 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.551965 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 08 15:18:47.556359 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.556340 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72"] Sep 08 15:18:47.562835 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.562808 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sw2zc"] Sep 08 15:18:47.566322 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.566303 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.569484 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.569464 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 08 15:18:47.569707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.569690 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 08 15:18:47.569904 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.569890 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 08 15:18:47.569987 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.569914 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fp5sn\"" Sep 08 15:18:47.571958 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.571935 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-stxmz"] Sep 08 15:18:47.603833 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.603797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/21c79cef-1555-4a97-be07-4a52e9a4b965-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.603833 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.603830 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgbds\" (UniqueName: \"kubernetes.io/projected/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-kube-api-access-hgbds\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.604145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.603854 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a069451-f309-4a43-85c5-8a1844704331-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.604145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.603874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-metrics-client-ca\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.604145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.603943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-sys\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.604145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.603981 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-wtmp\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.604145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604004 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.604145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.604145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604084 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkv6f\" (UniqueName: \"kubernetes.io/projected/9a069451-f309-4a43-85c5-8a1844704331-kube-api-access-qkv6f\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.604145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j488g\" (UniqueName: \"kubernetes.io/projected/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-api-access-j488g\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.604457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604156 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-root\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.604457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604196 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.604457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604217 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-textfile\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.604457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-accelerators-collector-config\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.604457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a069451-f309-4a43-85c5-8a1844704331-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.604457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604294 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-tls\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.604457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604360 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a069451-f309-4a43-85c5-8a1844704331-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.604457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604380 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21c79cef-1555-4a97-be07-4a52e9a4b965-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.604457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.604398 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.702036 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.702004 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b6g8j"] Sep 08 15:18:47.704807 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:47.704780 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1baffed3_ce48_4637_abcd_01dd21feed17.slice/crio-9f2e9c22b2160276560bd90364efa697c3a5eeb82b8d11377df4ac785da38106 WatchSource:0}: Error finding container 9f2e9c22b2160276560bd90364efa697c3a5eeb82b8d11377df4ac785da38106: Status 404 returned error can't find the container with id 9f2e9c22b2160276560bd90364efa697c3a5eeb82b8d11377df4ac785da38106 Sep 08 15:18:47.704916 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.704895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.704958 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.704935 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-textfile\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.705057 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-accelerators-collector-config\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.705122 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a069451-f309-4a43-85c5-8a1844704331-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.705179 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-tls\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.705228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a069451-f309-4a43-85c5-8a1844704331-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.705228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705211 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21c79cef-1555-4a97-be07-4a52e9a4b965-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.705328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705237 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.705328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705267 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-textfile\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.705328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/21c79cef-1555-4a97-be07-4a52e9a4b965-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.705724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgbds\" (UniqueName: \"kubernetes.io/projected/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-kube-api-access-hgbds\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.705724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705589 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a069451-f309-4a43-85c5-8a1844704331-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.705724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705623 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-metrics-client-ca\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.705724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-sys\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.705724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705686 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/21c79cef-1555-4a97-be07-4a52e9a4b965-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.705724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705703 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-wtmp\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.705724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705711 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-accelerators-collector-config\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.706083 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.705733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.706083 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706045 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21c79cef-1555-4a97-be07-4a52e9a4b965-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.706209 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706091 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.706209 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706096 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-sys\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.706209 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:47.706138 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 08 15:18:47.706352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706269 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-wtmp\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.706352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706315 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-metrics-client-ca\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.706352 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:47.706327 2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 08 15:18:47.706500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.706500 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:47.706403 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-tls podName:0c5dd6d3-221c-4795-90d4-9bc2526b7c8c nodeName:}" failed. No retries permitted until 2026-09-08 15:18:48.206360802 +0000 UTC m=+67.229985253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-tls") pod "node-exporter-sw2zc" (UID: "0c5dd6d3-221c-4795-90d4-9bc2526b7c8c") : secret "node-exporter-tls" not found Sep 08 15:18:47.706500 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:47.706445 2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 08 15:18:47.706651 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:47.706500 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a069451-f309-4a43-85c5-8a1844704331-openshift-state-metrics-tls podName:9a069451-f309-4a43-85c5-8a1844704331 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:48.206476991 +0000 UTC m=+67.230101438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9a069451-f309-4a43-85c5-8a1844704331-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-2tr72" (UID: "9a069451-f309-4a43-85c5-8a1844704331") : secret "openshift-state-metrics-tls" not found Sep 08 15:18:47.706651 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:47.706524 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-tls podName:21c79cef-1555-4a97-be07-4a52e9a4b965 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:48.206514172 +0000 UTC m=+67.230138607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-stxmz" (UID: "21c79cef-1555-4a97-be07-4a52e9a4b965") : secret "kube-state-metrics-tls" not found Sep 08 15:18:47.706651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkv6f\" (UniqueName: \"kubernetes.io/projected/9a069451-f309-4a43-85c5-8a1844704331-kube-api-access-qkv6f\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.706651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706586 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j488g\" (UniqueName: \"kubernetes.io/projected/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-api-access-j488g\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.706651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-root\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.706942 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.706694 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-root\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.707319 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.707296 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a069451-f309-4a43-85c5-8a1844704331-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.708287 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.708227 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.708702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.708674 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a069451-f309-4a43-85c5-8a1844704331-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.708898 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.708879 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.713579 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.713557 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c6lrg"] Sep 08 15:18:47.716479 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:47.716452 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63f55018_af6a_4631_8222_0a268c900911.slice/crio-35a00473930f41d540855f3d0571aa818eab285718f12b4c896b49fa0c0d3f69 WatchSource:0}: Error finding container 35a00473930f41d540855f3d0571aa818eab285718f12b4c896b49fa0c0d3f69: Status 404 returned error can't find the container with id 35a00473930f41d540855f3d0571aa818eab285718f12b4c896b49fa0c0d3f69 Sep 08 15:18:47.719076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.719058 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np4gb\"" Sep 08 15:18:47.719378 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.719345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkv6f\" (UniqueName: \"kubernetes.io/projected/9a069451-f309-4a43-85c5-8a1844704331-kube-api-access-qkv6f\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:47.722782 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.722761 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j488g\" (UniqueName: \"kubernetes.io/projected/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-api-access-j488g\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:47.722910 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.722895 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgbds\" (UniqueName: \"kubernetes.io/projected/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-kube-api-access-hgbds\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:47.726775 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.726717 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:47.755406 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.755370 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c6lrg" event={"ID":"63f55018-af6a-4631-8222-0a268c900911","Type":"ContainerStarted","Data":"35a00473930f41d540855f3d0571aa818eab285718f12b4c896b49fa0c0d3f69"} Sep 08 15:18:47.756603 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:47.756576 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b6g8j" event={"ID":"1baffed3-ce48-4637-abcd-01dd21feed17","Type":"ContainerStarted","Data":"9f2e9c22b2160276560bd90364efa697c3a5eeb82b8d11377df4ac785da38106"} Sep 08 15:18:48.021052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.020973 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c975x"] Sep 08 15:18:48.024499 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:48.024475 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe3fe39_80b5_4afc_8682_54fcabb2152d.slice/crio-51c320b0411dae239ebe0a335e5d781d1f2a38fc2eb7913687aa1260b6126048 WatchSource:0}: Error finding container 51c320b0411dae239ebe0a335e5d781d1f2a38fc2eb7913687aa1260b6126048: Status 404 returned error can't find the container with id 51c320b0411dae239ebe0a335e5d781d1f2a38fc2eb7913687aa1260b6126048 Sep 08 15:18:48.210261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.210223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:48.210744 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.210290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-tls\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:48.210744 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.210333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a069451-f309-4a43-85c5-8a1844704331-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:48.212799 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.212774 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c5dd6d3-221c-4795-90d4-9bc2526b7c8c-node-exporter-tls\") pod \"node-exporter-sw2zc\" (UID: \"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c\") " pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:48.212942 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.212920 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21c79cef-1555-4a97-be07-4a52e9a4b965-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-stxmz\" (UID: \"21c79cef-1555-4a97-be07-4a52e9a4b965\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:48.212997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.212966 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a069451-f309-4a43-85c5-8a1844704331-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-2tr72\" (UID: \"9a069451-f309-4a43-85c5-8a1844704331\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:48.452793 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.452757 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" Sep 08 15:18:48.459145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.458941 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" Sep 08 15:18:48.475591 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.475336 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sw2zc" Sep 08 15:18:48.493221 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:48.493179 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c5dd6d3_221c_4795_90d4_9bc2526b7c8c.slice/crio-bb53473b3da76e37bebf55efab796b82daa40a67c1ffa664d29ed15bda7ab19d WatchSource:0}: Error finding container bb53473b3da76e37bebf55efab796b82daa40a67c1ffa664d29ed15bda7ab19d: Status 404 returned error can't find the container with id bb53473b3da76e37bebf55efab796b82daa40a67c1ffa664d29ed15bda7ab19d Sep 08 15:18:48.758239 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.758143 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72"] Sep 08 15:18:48.763962 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.763885 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c975x" event={"ID":"9fe3fe39-80b5-4afc-8682-54fcabb2152d","Type":"ContainerStarted","Data":"51c320b0411dae239ebe0a335e5d781d1f2a38fc2eb7913687aa1260b6126048"} Sep 08 15:18:48.766348 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.766320 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sw2zc" event={"ID":"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c","Type":"ContainerStarted","Data":"bb53473b3da76e37bebf55efab796b82daa40a67c1ffa664d29ed15bda7ab19d"} Sep 08 15:18:48.817744 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.817444 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-stxmz"] Sep 08 15:18:48.855750 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.854930 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 15:18:48.861199 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.860173 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.872076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.871402 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 15:18:48.872076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.871443 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 15:18:48.872076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.871637 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 15:18:48.872076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.871710 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 15:18:48.872076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.871869 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 15:18:48.872076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.871919 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 15:18:48.873682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.872581 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 15:18:48.873682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.872666 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gv5nn\"" Sep 08 15:18:48.873682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.872887 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 15:18:48.873682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.873147 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 15:18:48.912130 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:48.912073 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21c79cef_1555_4a97_be07_4a52e9a4b965.slice/crio-67bdc7382be7d98ce9f0639f57eaca040c19702f7913888c8618d90bd1f6fa95 WatchSource:0}: Error finding container 67bdc7382be7d98ce9f0639f57eaca040c19702f7913888c8618d90bd1f6fa95: Status 404 returned error can't find the container with id 67bdc7382be7d98ce9f0639f57eaca040c19702f7913888c8618d90bd1f6fa95 Sep 08 15:18:48.912982 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:48.912954 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a069451_f309_4a43_85c5_8a1844704331.slice/crio-feba8e22d0aaeb19fbcdef60ac7a4a4c4509edd7cb362e5c3f28e14947e30fc2 WatchSource:0}: Error finding container feba8e22d0aaeb19fbcdef60ac7a4a4c4509edd7cb362e5c3f28e14947e30fc2: Status 404 returned error can't find the container with id feba8e22d0aaeb19fbcdef60ac7a4a4c4509edd7cb362e5c3f28e14947e30fc2 Sep 08 15:18:48.916258 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916278 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916513 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916513 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916419 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-config-volume\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916513 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmln2\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-kube-api-access-jmln2\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916513 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916475 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-config-out\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916513 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916749 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916523 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916749 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916546 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916749 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916585 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.916749 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.916611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-web-config\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:48.917880 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:48.917855 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.017992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-config-out\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018068 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018163 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018189 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-web-config\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018259 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018351 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-config-volume\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.019592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.018435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmln2\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-kube-api-access-jmln2\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.022645 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:18:49.020490 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-trusted-ca-bundle podName:bca47c51-d406-4beb-ad90-33f9519a806f nodeName:}" failed. No retries permitted until 2026-09-08 15:18:49.520464201 +0000 UTC m=+68.544088641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f") : configmap references non-existent config key: ca-bundle.crt Sep 08 15:18:49.029203 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.028601 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-config-volume\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.035811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.029561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.035811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.029841 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.035811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.030681 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.035811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.030705 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.035811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.032714 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.035811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.032932 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.035811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.033075 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-config-out\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.035811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.034984 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.037403 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.037331 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.039438 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.038779 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-web-config\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.039438 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.039261 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmln2\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-kube-api-access-jmln2\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.523711 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.523670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.524702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.524665 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.653402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.653368 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-794f578c9f-kcmjm"] Sep 08 15:18:49.658752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.658709 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.662183 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.661718 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 08 15:18:49.662183 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.661781 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kh9px\"" Sep 08 15:18:49.662183 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.661935 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 08 15:18:49.662412 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.662196 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9diaos0qohrkj\"" Sep 08 15:18:49.662412 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.662227 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 08 15:18:49.662779 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.662759 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 08 15:18:49.663608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.663282 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 08 15:18:49.671666 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.671523 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-794f578c9f-kcmjm"] Sep 08 15:18:49.725713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.725181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-grpc-tls\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.725713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.725313 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.725713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.725373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.725713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.725407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pbbn\" (UniqueName: \"kubernetes.io/projected/e042c070-b2f4-407a-adcb-28018c9572ed-kube-api-access-9pbbn\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.725713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.725451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.725713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.725482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.725713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.725528 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-tls\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.725713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.725557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e042c070-b2f4-407a-adcb-28018c9572ed-metrics-client-ca\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.775697 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.774783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c6lrg" event={"ID":"63f55018-af6a-4631-8222-0a268c900911","Type":"ContainerStarted","Data":"27c72d89e0287735f8baa8ae07dcbd89f14aaf22ad6efa6dd8f160b8a9d396c1"} Sep 08 15:18:49.775697 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.774856 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c6lrg" event={"ID":"63f55018-af6a-4631-8222-0a268c900911","Type":"ContainerStarted","Data":"d3593e5be2e628f84a031ec1f39c6abf489d7cf320fea80ec1b92849abfe7c2f"} Sep 08 15:18:49.776050 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.776024 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:18:49.777433 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.777400 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" event={"ID":"21c79cef-1555-4a97-be07-4a52e9a4b965","Type":"ContainerStarted","Data":"67bdc7382be7d98ce9f0639f57eaca040c19702f7913888c8618d90bd1f6fa95"} Sep 08 15:18:49.779806 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.779705 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" event={"ID":"9a069451-f309-4a43-85c5-8a1844704331","Type":"ContainerStarted","Data":"914e45b1d6ec0352f8b13de86980dfddebe2679be5557cfc617acdd7229eed34"} Sep 08 15:18:49.779806 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.779763 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" event={"ID":"9a069451-f309-4a43-85c5-8a1844704331","Type":"ContainerStarted","Data":"b2d95fac801f15dba3ab0215ea39fb3a27c52aa3e23b287d09e28eb4255e378f"} Sep 08 15:18:49.779806 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.779784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" event={"ID":"9a069451-f309-4a43-85c5-8a1844704331","Type":"ContainerStarted","Data":"feba8e22d0aaeb19fbcdef60ac7a4a4c4509edd7cb362e5c3f28e14947e30fc2"} Sep 08 15:18:49.794370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.794312 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c6lrg" podStartSLOduration=67.547238167 podStartE2EDuration="1m8.794290485s" totalImagesPullingTime="1.247052318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:49.793530875 +0000 UTC m=+68.817155345" watchObservedRunningTime="2026-09-08 15:18:49.794290485 +0000 UTC m=+68.817915000" Sep 08 15:18:49.826222 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.826182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-tls\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.826390 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.826234 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e042c070-b2f4-407a-adcb-28018c9572ed-metrics-client-ca\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.826390 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.826288 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-grpc-tls\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.826390 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.826336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.826529 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.826391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.826529 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.826421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pbbn\" (UniqueName: \"kubernetes.io/projected/e042c070-b2f4-407a-adcb-28018c9572ed-kube-api-access-9pbbn\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.826529 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.826485 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.826529 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.826515 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.828480 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.828448 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e042c070-b2f4-407a-adcb-28018c9572ed-metrics-client-ca\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.830564 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.830536 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.832880 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.832849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.833104 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.833080 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-grpc-tls\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.833209 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.833093 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.833284 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.833267 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-tls\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.835344 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.835298 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e042c070-b2f4-407a-adcb-28018c9572ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.837606 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.837563 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pbbn\" (UniqueName: \"kubernetes.io/projected/e042c070-b2f4-407a-adcb-28018c9572ed-kube-api-access-9pbbn\") pod \"thanos-querier-794f578c9f-kcmjm\" (UID: \"e042c070-b2f4-407a-adcb-28018c9572ed\") " pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:49.974194 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:49.974157 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:50.132763 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:50.132727 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 15:18:50.137739 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:50.137701 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca47c51_d406_4beb_ad90_33f9519a806f.slice/crio-7608592e1594fd2f04b5f577a48fb76f6de336aad2d1a65f1ea03b87b40b1971 WatchSource:0}: Error finding container 7608592e1594fd2f04b5f577a48fb76f6de336aad2d1a65f1ea03b87b40b1971: Status 404 returned error can't find the container with id 7608592e1594fd2f04b5f577a48fb76f6de336aad2d1a65f1ea03b87b40b1971 Sep 08 15:18:50.288581 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:50.288546 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-794f578c9f-kcmjm"] Sep 08 15:18:50.292470 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:50.292437 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode042c070_b2f4_407a_adcb_28018c9572ed.slice/crio-8e87eae154e9ad2a0baa04907555d2b72053be6cd5ac5caf4df93cf6f9c4b7cb WatchSource:0}: Error finding container 8e87eae154e9ad2a0baa04907555d2b72053be6cd5ac5caf4df93cf6f9c4b7cb: Status 404 returned error can't find the container with id 8e87eae154e9ad2a0baa04907555d2b72053be6cd5ac5caf4df93cf6f9c4b7cb Sep 08 15:18:50.789425 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:50.788613 2578 generic.go:356] "Generic (PLEG): container finished" podID="0c5dd6d3-221c-4795-90d4-9bc2526b7c8c" containerID="9959aec89abd5aad3a4d68d0678b26296c4c98e4c860d9235993c65e3b940000" exitCode=0 Sep 08 15:18:50.789425 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:50.788727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sw2zc" event={"ID":"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c","Type":"ContainerDied","Data":"9959aec89abd5aad3a4d68d0678b26296c4c98e4c860d9235993c65e3b940000"} Sep 08 15:18:50.791023 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:50.790935 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" event={"ID":"e042c070-b2f4-407a-adcb-28018c9572ed","Type":"ContainerStarted","Data":"8e87eae154e9ad2a0baa04907555d2b72053be6cd5ac5caf4df93cf6f9c4b7cb"} Sep 08 15:18:50.793169 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:50.793057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerStarted","Data":"7608592e1594fd2f04b5f577a48fb76f6de336aad2d1a65f1ea03b87b40b1971"} Sep 08 15:18:51.730538 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:51.730500 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-crrxx" Sep 08 15:18:52.099420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.099330 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp"] Sep 08 15:18:52.120465 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.120429 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.122234 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.122206 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp"] Sep 08 15:18:52.125854 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.125823 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 08 15:18:52.125996 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.125900 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4rnrj39tms5kd\"" Sep 08 15:18:52.125996 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.125905 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 08 15:18:52.126438 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.126421 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 08 15:18:52.126919 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.126900 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7n55d\"" Sep 08 15:18:52.128172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.127770 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 08 15:18:52.150022 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.149983 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-secret-metrics-server-client-certs\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.150022 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.150019 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8j4g\" (UniqueName: \"kubernetes.io/projected/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-kube-api-access-n8j4g\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.150266 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.150042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.150266 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.150102 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-secret-metrics-server-tls\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.150266 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.150144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-client-ca-bundle\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.150266 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.150202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-audit-log\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.150266 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.150220 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-metrics-server-audit-profiles\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.250684 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.250650 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-secret-metrics-server-client-certs\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.250684 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.250686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8j4g\" (UniqueName: \"kubernetes.io/projected/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-kube-api-access-n8j4g\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.250939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.250719 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.250939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.250773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-secret-metrics-server-tls\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.250939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.250800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-client-ca-bundle\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.251095 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.250987 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-audit-log\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.251095 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.251023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-metrics-server-audit-profiles\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.251458 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.251390 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-audit-log\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.251615 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.251595 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.252311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.252288 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-metrics-server-audit-profiles\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.253552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.253528 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-secret-metrics-server-tls\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.253881 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.253864 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-secret-metrics-server-client-certs\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.253919 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.253892 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-client-ca-bundle\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.259831 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.259803 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8j4g\" (UniqueName: \"kubernetes.io/projected/c44cbd2f-5c56-4baa-84aa-9cee6ae7a859-kube-api-access-n8j4g\") pod \"metrics-server-6fdbc6dc49-qvnxp\" (UID: \"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859\") " pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.431031 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.430939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:18:52.666939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.666901 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85b564cc54-jtt5r"] Sep 08 15:18:52.687707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.687626 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85b564cc54-jtt5r"] Sep 08 15:18:52.687855 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.687764 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.690603 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.690577 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 08 15:18:52.690603 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.690591 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 08 15:18:52.690796 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.690631 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 08 15:18:52.690796 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.690577 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 08 15:18:52.690796 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.690763 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 08 15:18:52.691064 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.691048 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2qp8w\"" Sep 08 15:18:52.696600 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.696577 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 08 15:18:52.755514 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.755477 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa349055-1cae-4668-a546-e6bcab9adabd-metrics-client-ca\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.755678 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.755544 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-secret-telemeter-client\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.755678 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.755573 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa349055-1cae-4668-a546-e6bcab9adabd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.755678 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.755620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa349055-1cae-4668-a546-e6bcab9adabd-serving-certs-ca-bundle\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.755678 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.755665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-telemeter-client-tls\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.755858 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.755700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-federate-client-tls\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.755858 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.755724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.755858 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.755743 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clgh8\" (UniqueName: \"kubernetes.io/projected/aa349055-1cae-4668-a546-e6bcab9adabd-kube-api-access-clgh8\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.856749 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.856706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-telemeter-client-tls\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.856943 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.856771 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-federate-client-tls\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.856943 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.856802 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.856943 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.856831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clgh8\" (UniqueName: \"kubernetes.io/projected/aa349055-1cae-4668-a546-e6bcab9adabd-kube-api-access-clgh8\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.856943 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.856879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa349055-1cae-4668-a546-e6bcab9adabd-metrics-client-ca\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.856943 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.856936 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-secret-telemeter-client\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.857224 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.856955 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa349055-1cae-4668-a546-e6bcab9adabd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.857224 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.856983 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa349055-1cae-4668-a546-e6bcab9adabd-serving-certs-ca-bundle\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.857748 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.857712 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa349055-1cae-4668-a546-e6bcab9adabd-serving-certs-ca-bundle\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.857876 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.857813 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa349055-1cae-4668-a546-e6bcab9adabd-metrics-client-ca\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.858127 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.858082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa349055-1cae-4668-a546-e6bcab9adabd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.860038 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.860011 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.860257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.860228 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-federate-client-tls\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.860373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.860356 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-telemeter-client-tls\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.860448 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.860397 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aa349055-1cae-4668-a546-e6bcab9adabd-secret-telemeter-client\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:52.866824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:52.866797 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clgh8\" (UniqueName: \"kubernetes.io/projected/aa349055-1cae-4668-a546-e6bcab9adabd-kube-api-access-clgh8\") pod \"telemeter-client-85b564cc54-jtt5r\" (UID: \"aa349055-1cae-4668-a546-e6bcab9adabd\") " pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:53.000153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:53.000089 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" Sep 08 15:18:55.319714 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.319604 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp"] Sep 08 15:18:55.361537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.360270 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85b564cc54-jtt5r"] Sep 08 15:18:55.373918 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:55.373879 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa349055_1cae_4668_a546_e6bcab9adabd.slice/crio-59292e7a98f0638576682de2829a0945c171729ba63ee0f48c0c98d3c5cec27d WatchSource:0}: Error finding container 59292e7a98f0638576682de2829a0945c171729ba63ee0f48c0c98d3c5cec27d: Status 404 returned error can't find the container with id 59292e7a98f0638576682de2829a0945c171729ba63ee0f48c0c98d3c5cec27d Sep 08 15:18:55.817428 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.817387 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sw2zc" event={"ID":"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c","Type":"ContainerStarted","Data":"d322bdfef5d4451ba4e82b7d1d08336811e7a8e0261eb84e44490239e052165c"} Sep 08 15:18:55.817625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.817452 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sw2zc" event={"ID":"0c5dd6d3-221c-4795-90d4-9bc2526b7c8c","Type":"ContainerStarted","Data":"aae8a9b62f8830f4053a1c380a27dc30f55095b495d31f01f6d03ba9bf8359ae"} Sep 08 15:18:55.819493 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.819469 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" event={"ID":"21c79cef-1555-4a97-be07-4a52e9a4b965","Type":"ContainerStarted","Data":"6fbfb1a5c090445b3911a5294f80fd5956f38d0de423d77e03366fed8c6b6324"} Sep 08 15:18:55.819592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.819522 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" event={"ID":"21c79cef-1555-4a97-be07-4a52e9a4b965","Type":"ContainerStarted","Data":"cdc088612f65e4c156d7c8cb7e51173f5a59051857766e29e404b4d7f3172039"} Sep 08 15:18:55.819592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.819543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" event={"ID":"21c79cef-1555-4a97-be07-4a52e9a4b965","Type":"ContainerStarted","Data":"9d2d3da97230bc842fe51495b6306c75fa9c24a5686e1723f632d08cf4941d98"} Sep 08 15:18:55.821180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.821154 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b6g8j" event={"ID":"1baffed3-ce48-4637-abcd-01dd21feed17","Type":"ContainerStarted","Data":"220d088743da2811c8edf4b7ed63f62e695ea9370e4b748446865efb37c17f9e"} Sep 08 15:18:55.822315 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.822289 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" event={"ID":"aa349055-1cae-4668-a546-e6bcab9adabd","Type":"ContainerStarted","Data":"59292e7a98f0638576682de2829a0945c171729ba63ee0f48c0c98d3c5cec27d"} Sep 08 15:18:55.824472 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.824453 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" event={"ID":"9a069451-f309-4a43-85c5-8a1844704331","Type":"ContainerStarted","Data":"142d43bb2421caef30fd1cd181deb33ccf0300520cc477fbfb420f1b461116f6"} Sep 08 15:18:55.826477 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.826449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" event={"ID":"e042c070-b2f4-407a-adcb-28018c9572ed","Type":"ContainerStarted","Data":"0b4dfb2e9012a0ca97b412696c35d880d96f9397cca2233e898b5b614fe827fd"} Sep 08 15:18:55.826562 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.826512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" event={"ID":"e042c070-b2f4-407a-adcb-28018c9572ed","Type":"ContainerStarted","Data":"b8f570fa3bfc7b1915b99f7cf8d6c08f71fd40466a4c617afe3584a660c98557"} Sep 08 15:18:55.826562 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.826538 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" event={"ID":"e042c070-b2f4-407a-adcb-28018c9572ed","Type":"ContainerStarted","Data":"73c58ac895ff49c229d9bf007a22b6b89df356d03335bb2f350c2528f85b1343"} Sep 08 15:18:55.828013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.827990 2578 generic.go:356] "Generic (PLEG): container finished" podID="bca47c51-d406-4beb-ad90-33f9519a806f" containerID="ba70f0efaf9a15fcc78ec57b20c37a65cc3b963abb8411501b863ce4c64d54ca" exitCode=0 Sep 08 15:18:55.828102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.828066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerDied","Data":"ba70f0efaf9a15fcc78ec57b20c37a65cc3b963abb8411501b863ce4c64d54ca"} Sep 08 15:18:55.829273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.829251 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" event={"ID":"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859","Type":"ContainerStarted","Data":"58e242b83fb659731f05ef0339263f06c5bdc6a3474df3a4027c1646c7b376f7"} Sep 08 15:18:55.830795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.830775 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c975x" event={"ID":"9fe3fe39-80b5-4afc-8682-54fcabb2152d","Type":"ContainerStarted","Data":"8c29cdec85f56aa957624aadb461cf9cef3b8c3526ccfaf0e1c558c477e0faf1"} Sep 08 15:18:55.830960 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.830890 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:18:55.844497 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.843494 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sw2zc" podStartSLOduration=7.371073222 podStartE2EDuration="8.843473627s" totalImagesPullingTime="1.372781614s" totalInitContainerRuntime="99.618791ms" isStatefulPod=true podCreationTimestamp="2026-09-08 15:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:55.843069695 +0000 UTC m=+74.866694164" watchObservedRunningTime="2026-09-08 15:18:55.843473627 +0000 UTC m=+74.867098076" Sep 08 15:18:55.896248 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.896172 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-stxmz" podStartSLOduration=2.83137791 podStartE2EDuration="8.896151528s" totalImagesPullingTime="6.064773618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:55.893705386 +0000 UTC m=+74.917329841" watchObservedRunningTime="2026-09-08 15:18:55.896151528 +0000 UTC m=+74.919775984" Sep 08 15:18:55.913328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.913274 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c975x" podStartSLOduration=67.898577104 podStartE2EDuration="1m14.913258359s" totalImagesPullingTime="7.014681255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:55.912170281 +0000 UTC m=+74.935794737" watchObservedRunningTime="2026-09-08 15:18:55.913258359 +0000 UTC m=+74.936882814" Sep 08 15:18:55.929894 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.929838 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b6g8j" podStartSLOduration=67.599988323 podStartE2EDuration="1m14.92982349s" totalImagesPullingTime="7.329835167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:55.928066335 +0000 UTC m=+74.951690806" watchObservedRunningTime="2026-09-08 15:18:55.92982349 +0000 UTC m=+74.953447950" Sep 08 15:18:55.948766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:55.948713 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-2tr72" podStartSLOduration=2.994644856 podStartE2EDuration="8.948690032s" totalImagesPullingTime="5.954045176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:55.947440435 +0000 UTC m=+74.971064890" watchObservedRunningTime="2026-09-08 15:18:55.948690032 +0000 UTC m=+74.972314488" Sep 08 15:18:56.371608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.371574 2578 patch_prober.go:28] interesting pod/image-registry-6866fd6c9c-fbltn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 08 15:18:56.371999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.371630 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" podUID="1b72473a-b0b1-4b67-913f-930bfffbbda0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 15:18:56.487532 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.486982 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-cnffw"] Sep 08 15:18:56.490444 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.490387 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-cnffw" Sep 08 15:18:56.493138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.493036 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2nwzz\"" Sep 08 15:18:56.493543 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.493523 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 08 15:18:56.493774 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.493637 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 08 15:18:56.502019 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.501989 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-cnffw"] Sep 08 15:18:56.607873 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.607787 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hpt9\" (UniqueName: \"kubernetes.io/projected/c3db4cab-73ab-45a2-9b43-ea0aa0bcb93c-kube-api-access-9hpt9\") pod \"downloads-85d98c998f-cnffw\" (UID: \"c3db4cab-73ab-45a2-9b43-ea0aa0bcb93c\") " pod="openshift-console/downloads-85d98c998f-cnffw" Sep 08 15:18:56.708847 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.708807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hpt9\" (UniqueName: \"kubernetes.io/projected/c3db4cab-73ab-45a2-9b43-ea0aa0bcb93c-kube-api-access-9hpt9\") pod \"downloads-85d98c998f-cnffw\" (UID: \"c3db4cab-73ab-45a2-9b43-ea0aa0bcb93c\") " pod="openshift-console/downloads-85d98c998f-cnffw" Sep 08 15:18:56.718648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.718619 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hpt9\" (UniqueName: \"kubernetes.io/projected/c3db4cab-73ab-45a2-9b43-ea0aa0bcb93c-kube-api-access-9hpt9\") pod \"downloads-85d98c998f-cnffw\" (UID: \"c3db4cab-73ab-45a2-9b43-ea0aa0bcb93c\") " pod="openshift-console/downloads-85d98c998f-cnffw" Sep 08 15:18:56.803970 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.803928 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-cnffw" Sep 08 15:18:56.842094 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.842055 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" event={"ID":"e042c070-b2f4-407a-adcb-28018c9572ed","Type":"ContainerStarted","Data":"89e52852c6f229796a187dd9bb127a076ea62fefaceefa15938bcaed687d997e"} Sep 08 15:18:56.844352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.842158 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" event={"ID":"e042c070-b2f4-407a-adcb-28018c9572ed","Type":"ContainerStarted","Data":"b0e1930aff445f1cf3f6da3b91d4380d50dfa9207f710b247b053d3aa4e449b8"} Sep 08 15:18:56.844352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.842201 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" event={"ID":"e042c070-b2f4-407a-adcb-28018c9572ed","Type":"ContainerStarted","Data":"19b18f9e94e6a052a7590c2c08cfcd5880fc8fbd504455c77dd0f6ee2b3733a3"} Sep 08 15:18:56.844352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.843544 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:18:56.867005 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:56.866890 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" podStartSLOduration=2.270007431 podStartE2EDuration="7.866864525s" totalImagesPullingTime="5.596857094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:56.865823824 +0000 UTC m=+75.889448308" watchObservedRunningTime="2026-09-08 15:18:56.866864525 +0000 UTC m=+75.890488980" Sep 08 15:18:57.852276 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:57.852048 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerStarted","Data":"542009a4f76d265b06ee3929f9e0e8eb590b2a94e45cfbd203d7d4b874ab5346"} Sep 08 15:18:57.852276 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:57.852202 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerStarted","Data":"174b39b032d6f812394e5415996536daf503e455dd9d4241248c5d94a9214f89"} Sep 08 15:18:57.858988 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:57.856158 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" event={"ID":"aa349055-1cae-4668-a546-e6bcab9adabd","Type":"ContainerStarted","Data":"c76ae9a4a608d81e6d92901dd75a7f62b4ddaa7bea1782cdabd6180a83091905"} Sep 08 15:18:57.858988 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:57.856221 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" event={"ID":"aa349055-1cae-4668-a546-e6bcab9adabd","Type":"ContainerStarted","Data":"8b8ef05f4616a8bc005ff0928fe8a53bfd0e75f97612c8a9aa4de4be933b8dde"} Sep 08 15:18:57.940217 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:57.940142 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-cnffw"] Sep 08 15:18:57.946051 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:18:57.946023 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3db4cab_73ab_45a2_9b43_ea0aa0bcb93c.slice/crio-2d4bd2eda194b897695777212ea8ea286c512cefa261b5cfc796ff609c1ba4d5 WatchSource:0}: Error finding container 2d4bd2eda194b897695777212ea8ea286c512cefa261b5cfc796ff609c1ba4d5: Status 404 returned error can't find the container with id 2d4bd2eda194b897695777212ea8ea286c512cefa261b5cfc796ff609c1ba4d5 Sep 08 15:18:58.709433 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.709407 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6866fd6c9c-fbltn" Sep 08 15:18:58.864569 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.864513 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" event={"ID":"aa349055-1cae-4668-a546-e6bcab9adabd","Type":"ContainerStarted","Data":"b6855c0d3def0ed70fa292649ee528e65c4bbea679c535bc65fd4f4992ae08cc"} Sep 08 15:18:58.869183 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.869148 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerStarted","Data":"7416998cf6ba7e05201f558372bb749042b5e064f86cf727d73d569932a98bec"} Sep 08 15:18:58.869340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.869233 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerStarted","Data":"44a8875260503ad89e1c7589e3f27362509c7bbc3423d1cab403aaec39fdef84"} Sep 08 15:18:58.869340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.869277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerStarted","Data":"6874630234c230540099119202f134a5416b403159d0c080dfaf9ef1410d671e"} Sep 08 15:18:58.869340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.869316 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerStarted","Data":"641205b53a04fdbab46707375fc9a80318d34b99d749446f788ddf0b2c760915"} Sep 08 15:18:58.871559 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.871531 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" event={"ID":"c44cbd2f-5c56-4baa-84aa-9cee6ae7a859","Type":"ContainerStarted","Data":"773d8b989827dce8c1e58e8355c69b3432bac0289242058b42a294f9dd91283a"} Sep 08 15:18:58.873942 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.873890 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-cnffw" event={"ID":"c3db4cab-73ab-45a2-9b43-ea0aa0bcb93c","Type":"ContainerStarted","Data":"2d4bd2eda194b897695777212ea8ea286c512cefa261b5cfc796ff609c1ba4d5"} Sep 08 15:18:58.906292 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.906231 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85b564cc54-jtt5r" podStartSLOduration=4.62457009 podStartE2EDuration="6.906209036s" totalImagesPullingTime="2.281638946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:58.903830728 +0000 UTC m=+77.927455187" watchObservedRunningTime="2026-09-08 15:18:58.906209036 +0000 UTC m=+77.929833495" Sep 08 15:18:58.953625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.953570 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.172322053 podStartE2EDuration="10.953550098s" totalImagesPullingTime="6.750295029s" totalInitContainerRuntime="30.933016ms" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:58.952142568 +0000 UTC m=+77.975767025" watchObservedRunningTime="2026-09-08 15:18:58.953550098 +0000 UTC m=+77.977174554" Sep 08 15:18:58.999930 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:18:58.999863 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" podStartSLOduration=3.6429291409999998 podStartE2EDuration="6.999837135s" totalImagesPullingTime="3.356907994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:58.998495657 +0000 UTC m=+78.022120124" watchObservedRunningTime="2026-09-08 15:18:58.999837135 +0000 UTC m=+78.023461804" Sep 08 15:19:01.135367 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.135335 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c959f7bcf-85kd2"] Sep 08 15:19:01.138647 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.138628 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.145743 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.145534 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 08 15:19:01.145743 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.145548 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 08 15:19:01.145743 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.145567 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 08 15:19:01.145743 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.145598 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 08 15:19:01.145743 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.145538 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-52v47\"" Sep 08 15:19:01.146055 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.145879 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 08 15:19:01.154636 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.154602 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c959f7bcf-85kd2"] Sep 08 15:19:01.259348 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.259309 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-serving-cert\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.259552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.259450 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-config\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.259552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.259489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-service-ca\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.259552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.259527 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsm9t\" (UniqueName: \"kubernetes.io/projected/442f5a3e-06f5-4961-8dfa-b62b185ef003-kube-api-access-rsm9t\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.259721 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.259566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-oauth-config\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.259721 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.259586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-oauth-serving-cert\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.361013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.360943 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-config\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.361013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.361002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-service-ca\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.361288 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.361050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsm9t\" (UniqueName: \"kubernetes.io/projected/442f5a3e-06f5-4961-8dfa-b62b185ef003-kube-api-access-rsm9t\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.361288 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.361086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-oauth-config\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.361288 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.361138 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-oauth-serving-cert\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.361288 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.361176 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-serving-cert\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.361858 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.361798 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-service-ca\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.361858 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.361819 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-config\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.362235 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.362015 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-oauth-serving-cert\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.364178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.363911 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-oauth-config\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.364178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.364142 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-serving-cert\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.370566 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.370537 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsm9t\" (UniqueName: \"kubernetes.io/projected/442f5a3e-06f5-4961-8dfa-b62b185ef003-kube-api-access-rsm9t\") pod \"console-c959f7bcf-85kd2\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.448583 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.448492 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:01.739049 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.739016 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c959f7bcf-85kd2"] Sep 08 15:19:01.743436 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:19:01.743400 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod442f5a3e_06f5_4961_8dfa_b62b185ef003.slice/crio-7ac9d7629d2cf27cfcdbd264b2bc5e7f21261da92e2bf727d3b4d3bab8291111 WatchSource:0}: Error finding container 7ac9d7629d2cf27cfcdbd264b2bc5e7f21261da92e2bf727d3b4d3bab8291111: Status 404 returned error can't find the container with id 7ac9d7629d2cf27cfcdbd264b2bc5e7f21261da92e2bf727d3b4d3bab8291111 Sep 08 15:19:01.893505 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:01.893462 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c959f7bcf-85kd2" event={"ID":"442f5a3e-06f5-4961-8dfa-b62b185ef003","Type":"ContainerStarted","Data":"7ac9d7629d2cf27cfcdbd264b2bc5e7f21261da92e2bf727d3b4d3bab8291111"} Sep 08 15:19:03.882162 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:03.882103 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-794f578c9f-kcmjm" Sep 08 15:19:05.921066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:05.921029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c959f7bcf-85kd2" event={"ID":"442f5a3e-06f5-4961-8dfa-b62b185ef003","Type":"ContainerStarted","Data":"cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79"} Sep 08 15:19:05.951258 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:05.951200 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c959f7bcf-85kd2" podStartSLOduration=1.298065186 podStartE2EDuration="4.951180792s" totalImagesPullingTime="3.653115606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:19:05.949007702 +0000 UTC m=+84.972632155" watchObservedRunningTime="2026-09-08 15:19:05.951180792 +0000 UTC m=+84.974805248" Sep 08 15:19:09.883359 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:09.883321 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59db85fd6c-7l975"] Sep 08 15:19:09.919376 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:09.919337 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:09.933931 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:09.933899 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59db85fd6c-7l975"] Sep 08 15:19:09.937760 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:09.937737 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 08 15:19:10.047318 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.047270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-oauth-config\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.047318 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.047325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-trusted-ca-bundle\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.047549 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.047471 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-service-ca\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.047549 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.047534 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvjh7\" (UniqueName: \"kubernetes.io/projected/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-kube-api-access-kvjh7\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.047634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.047580 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-serving-cert\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.047634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.047612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-config\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.047711 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.047639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-oauth-serving-cert\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.148692 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.148590 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-service-ca\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.148692 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.148649 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvjh7\" (UniqueName: \"kubernetes.io/projected/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-kube-api-access-kvjh7\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.148692 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.148683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-serving-cert\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.148970 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.148710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-config\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.148970 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.148734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-oauth-serving-cert\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.148970 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.148896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-oauth-config\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.148970 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.148950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-trusted-ca-bundle\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.149496 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.149465 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-config\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.149621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.149514 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-service-ca\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.149621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.149541 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-oauth-serving-cert\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.149998 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.149976 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-trusted-ca-bundle\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.151673 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.151633 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-serving-cert\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.151805 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.151783 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-oauth-config\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.161968 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.161939 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvjh7\" (UniqueName: \"kubernetes.io/projected/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-kube-api-access-kvjh7\") pod \"console-59db85fd6c-7l975\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:10.230766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:10.230731 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:11.449195 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:11.449156 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:11.449645 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:11.449232 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:11.454489 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:11.454460 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:11.953615 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:11.953584 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:12.432170 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:12.432067 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:19:12.432170 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:12.432155 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:19:15.877765 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:15.877737 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59db85fd6c-7l975"] Sep 08 15:19:15.959223 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:19:15.959187 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd28b2da_69b8_4dc0_be9d_e0dd6cf19770.slice/crio-512a10feb3b80d761a7bd4fb8fa1a5ec61733d02616c1814af82af5262c4cff7 WatchSource:0}: Error finding container 512a10feb3b80d761a7bd4fb8fa1a5ec61733d02616c1814af82af5262c4cff7: Status 404 returned error can't find the container with id 512a10feb3b80d761a7bd4fb8fa1a5ec61733d02616c1814af82af5262c4cff7 Sep 08 15:19:15.967747 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:15.967715 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59db85fd6c-7l975" event={"ID":"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770","Type":"ContainerStarted","Data":"512a10feb3b80d761a7bd4fb8fa1a5ec61733d02616c1814af82af5262c4cff7"} Sep 08 15:19:16.975133 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:16.975047 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-cnffw" event={"ID":"c3db4cab-73ab-45a2-9b43-ea0aa0bcb93c","Type":"ContainerStarted","Data":"f2ad0dc73527b9a6e1f4a9a066927709ab87f6bea8a2adf48c91e691cb779f18"} Sep 08 15:19:16.975644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:16.975169 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-cnffw" Sep 08 15:19:16.977369 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:16.977313 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59db85fd6c-7l975" event={"ID":"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770","Type":"ContainerStarted","Data":"b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7"} Sep 08 15:19:16.992193 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:16.992161 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-cnffw" Sep 08 15:19:16.995811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:16.995753 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-cnffw" podStartSLOduration=2.911876096 podStartE2EDuration="20.995731039s" totalImagesPullingTime="18.083854943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:19:16.994676024 +0000 UTC m=+96.018300480" watchObservedRunningTime="2026-09-08 15:19:16.995731039 +0000 UTC m=+96.019355542" Sep 08 15:19:17.039061 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:17.038996 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59db85fd6c-7l975" podStartSLOduration=8.038970351 podStartE2EDuration="8.038970351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:19:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:19:17.036587559 +0000 UTC m=+96.060212038" watchObservedRunningTime="2026-09-08 15:19:17.038970351 +0000 UTC m=+96.062594807" Sep 08 15:19:20.231857 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:20.231810 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:20.232354 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:20.231888 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:20.237732 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:20.237700 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:21.008627 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:21.008594 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:19:21.086294 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:21.086256 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c959f7bcf-85kd2"] Sep 08 15:19:26.844951 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:26.844922 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c975x" Sep 08 15:19:32.437831 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:32.437799 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:19:32.442130 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:32.442088 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6fdbc6dc49-qvnxp" Sep 08 15:19:46.115824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.115761 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c959f7bcf-85kd2" podUID="442f5a3e-06f5-4961-8dfa-b62b185ef003" containerName="console" containerID="cri-o://cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79" gracePeriod=15 Sep 08 15:19:46.426997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.426957 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:46.592355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.591414 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-config\") pod \"442f5a3e-06f5-4961-8dfa-b62b185ef003\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " Sep 08 15:19:46.592355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.591481 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-serving-cert\") pod \"442f5a3e-06f5-4961-8dfa-b62b185ef003\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " Sep 08 15:19:46.592355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.591535 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-service-ca\") pod \"442f5a3e-06f5-4961-8dfa-b62b185ef003\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " Sep 08 15:19:46.592355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.591573 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsm9t\" (UniqueName: \"kubernetes.io/projected/442f5a3e-06f5-4961-8dfa-b62b185ef003-kube-api-access-rsm9t\") pod \"442f5a3e-06f5-4961-8dfa-b62b185ef003\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " Sep 08 15:19:46.592355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.591603 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-oauth-serving-cert\") pod \"442f5a3e-06f5-4961-8dfa-b62b185ef003\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " Sep 08 15:19:46.592355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.591696 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-oauth-config\") pod \"442f5a3e-06f5-4961-8dfa-b62b185ef003\" (UID: \"442f5a3e-06f5-4961-8dfa-b62b185ef003\") " Sep 08 15:19:46.593214 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.593160 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-service-ca" (OuterVolumeSpecName: "service-ca") pod "442f5a3e-06f5-4961-8dfa-b62b185ef003" (UID: "442f5a3e-06f5-4961-8dfa-b62b185ef003"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:19:46.593698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.593666 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "442f5a3e-06f5-4961-8dfa-b62b185ef003" (UID: "442f5a3e-06f5-4961-8dfa-b62b185ef003"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:19:46.593935 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.593896 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-config" (OuterVolumeSpecName: "console-config") pod "442f5a3e-06f5-4961-8dfa-b62b185ef003" (UID: "442f5a3e-06f5-4961-8dfa-b62b185ef003"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:19:46.598714 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.598208 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "442f5a3e-06f5-4961-8dfa-b62b185ef003" (UID: "442f5a3e-06f5-4961-8dfa-b62b185ef003"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:19:46.598714 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.598656 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/442f5a3e-06f5-4961-8dfa-b62b185ef003-kube-api-access-rsm9t" (OuterVolumeSpecName: "kube-api-access-rsm9t") pod "442f5a3e-06f5-4961-8dfa-b62b185ef003" (UID: "442f5a3e-06f5-4961-8dfa-b62b185ef003"). InnerVolumeSpecName "kube-api-access-rsm9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:19:46.600775 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.600736 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "442f5a3e-06f5-4961-8dfa-b62b185ef003" (UID: "442f5a3e-06f5-4961-8dfa-b62b185ef003"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:19:46.692789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.692588 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-oauth-config\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:19:46.692789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.692630 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-config\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:19:46.692789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.692647 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/442f5a3e-06f5-4961-8dfa-b62b185ef003-console-serving-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:19:46.692789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.692663 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:19:46.692789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.692679 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsm9t\" (UniqueName: \"kubernetes.io/projected/442f5a3e-06f5-4961-8dfa-b62b185ef003-kube-api-access-rsm9t\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:19:46.692789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:46.692694 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/442f5a3e-06f5-4961-8dfa-b62b185ef003-oauth-serving-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:19:47.121133 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.121076 2578 generic.go:356] "Generic (PLEG): container finished" podID="442f5a3e-06f5-4961-8dfa-b62b185ef003" containerID="cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79" exitCode=0 Sep 08 15:19:47.121574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.121150 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c959f7bcf-85kd2" Sep 08 15:19:47.121574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.121148 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c959f7bcf-85kd2" event={"ID":"442f5a3e-06f5-4961-8dfa-b62b185ef003","Type":"ContainerDied","Data":"cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79"} Sep 08 15:19:47.121574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.121273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c959f7bcf-85kd2" event={"ID":"442f5a3e-06f5-4961-8dfa-b62b185ef003","Type":"ContainerDied","Data":"7ac9d7629d2cf27cfcdbd264b2bc5e7f21261da92e2bf727d3b4d3bab8291111"} Sep 08 15:19:47.121574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.121304 2578 scope.go:117] "RemoveContainer" containerID="cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79" Sep 08 15:19:47.130984 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.130966 2578 scope.go:117] "RemoveContainer" containerID="cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79" Sep 08 15:19:47.131329 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:19:47.131305 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79\": container with ID starting with cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79 not found: ID does not exist" containerID="cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79" Sep 08 15:19:47.131441 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.131335 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79"} err="failed to get container status \"cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79\": rpc error: code = NotFound desc = could not find container \"cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79\": container with ID starting with cc12b9c241370bb82ec78604dc2626084d05f437a348b0153211598170721b79 not found: ID does not exist" Sep 08 15:19:47.151182 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.149411 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c959f7bcf-85kd2"] Sep 08 15:19:47.157514 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.157476 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c959f7bcf-85kd2"] Sep 08 15:19:47.501171 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:19:47.501139 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="442f5a3e-06f5-4961-8dfa-b62b185ef003" path="/var/lib/kubelet/pods/442f5a3e-06f5-4961-8dfa-b62b185ef003/volumes" Sep 08 15:20:08.009573 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.009537 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 15:20:08.010063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.009837 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="alertmanager" containerID="cri-o://174b39b032d6f812394e5415996536daf503e455dd9d4241248c5d94a9214f89" gracePeriod=120 Sep 08 15:20:08.010063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.009898 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy-metric" containerID="cri-o://44a8875260503ad89e1c7589e3f27362509c7bbc3423d1cab403aaec39fdef84" gracePeriod=120 Sep 08 15:20:08.010063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.009924 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="prom-label-proxy" containerID="cri-o://7416998cf6ba7e05201f558372bb749042b5e064f86cf727d73d569932a98bec" gracePeriod=120 Sep 08 15:20:08.010063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.009968 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="config-reloader" containerID="cri-o://542009a4f76d265b06ee3929f9e0e8eb590b2a94e45cfbd203d7d4b874ab5346" gracePeriod=120 Sep 08 15:20:08.010063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.009937 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy-web" containerID="cri-o://641205b53a04fdbab46707375fc9a80318d34b99d749446f788ddf0b2c760915" gracePeriod=120 Sep 08 15:20:08.010063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.009978 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy" containerID="cri-o://6874630234c230540099119202f134a5416b403159d0c080dfaf9ef1410d671e" gracePeriod=120 Sep 08 15:20:08.213531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213497 2578 generic.go:356] "Generic (PLEG): container finished" podID="bca47c51-d406-4beb-ad90-33f9519a806f" containerID="7416998cf6ba7e05201f558372bb749042b5e064f86cf727d73d569932a98bec" exitCode=0 Sep 08 15:20:08.213531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213522 2578 generic.go:356] "Generic (PLEG): container finished" podID="bca47c51-d406-4beb-ad90-33f9519a806f" containerID="44a8875260503ad89e1c7589e3f27362509c7bbc3423d1cab403aaec39fdef84" exitCode=0 Sep 08 15:20:08.213531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213528 2578 generic.go:356] "Generic (PLEG): container finished" podID="bca47c51-d406-4beb-ad90-33f9519a806f" containerID="6874630234c230540099119202f134a5416b403159d0c080dfaf9ef1410d671e" exitCode=0 Sep 08 15:20:08.213531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213534 2578 generic.go:356] "Generic (PLEG): container finished" podID="bca47c51-d406-4beb-ad90-33f9519a806f" containerID="542009a4f76d265b06ee3929f9e0e8eb590b2a94e45cfbd203d7d4b874ab5346" exitCode=0 Sep 08 15:20:08.213531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213539 2578 generic.go:356] "Generic (PLEG): container finished" podID="bca47c51-d406-4beb-ad90-33f9519a806f" containerID="174b39b032d6f812394e5415996536daf503e455dd9d4241248c5d94a9214f89" exitCode=0 Sep 08 15:20:08.213873 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213571 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerDied","Data":"7416998cf6ba7e05201f558372bb749042b5e064f86cf727d73d569932a98bec"} Sep 08 15:20:08.213873 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213652 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerDied","Data":"44a8875260503ad89e1c7589e3f27362509c7bbc3423d1cab403aaec39fdef84"} Sep 08 15:20:08.213873 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213692 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerDied","Data":"6874630234c230540099119202f134a5416b403159d0c080dfaf9ef1410d671e"} Sep 08 15:20:08.213873 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerDied","Data":"542009a4f76d265b06ee3929f9e0e8eb590b2a94e45cfbd203d7d4b874ab5346"} Sep 08 15:20:08.213873 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:08.213769 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerDied","Data":"174b39b032d6f812394e5415996536daf503e455dd9d4241248c5d94a9214f89"} Sep 08 15:20:09.222264 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.222233 2578 generic.go:356] "Generic (PLEG): container finished" podID="bca47c51-d406-4beb-ad90-33f9519a806f" containerID="641205b53a04fdbab46707375fc9a80318d34b99d749446f788ddf0b2c760915" exitCode=0 Sep 08 15:20:09.222641 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.222303 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerDied","Data":"641205b53a04fdbab46707375fc9a80318d34b99d749446f788ddf0b2c760915"} Sep 08 15:20:09.268351 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.268323 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:09.279555 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279522 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.279742 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279573 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmln2\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-kube-api-access-jmln2\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.279742 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279591 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-main-db\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.279742 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279621 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-cluster-tls-config\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.279888 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279741 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-config-volume\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.279888 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279794 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.279888 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279824 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-main-tls\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.279888 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279854 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-metrics-client-ca\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.280060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279924 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-web\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.280060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279930 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:20:09.280060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279952 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-config-out\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.280060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.279988 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-tls-assets\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.280060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.280027 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-web-config\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.280060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.280058 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-trusted-ca-bundle\") pod \"bca47c51-d406-4beb-ad90-33f9519a806f\" (UID: \"bca47c51-d406-4beb-ad90-33f9519a806f\") " Sep 08 15:20:09.282768 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.280497 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:20:09.282768 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.280531 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-main-db\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.282768 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.280959 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:20:09.283447 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.283098 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-kube-api-access-jmln2" (OuterVolumeSpecName: "kube-api-access-jmln2") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "kube-api-access-jmln2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:20:09.283447 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.283414 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:20:09.283619 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.283523 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:20:09.283905 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.283869 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-config-volume" (OuterVolumeSpecName: "config-volume") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:20:09.283986 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.283954 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:20:09.285011 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.284981 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:20:09.285319 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.285284 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:20:09.286237 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.286210 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-config-out" (OuterVolumeSpecName: "config-out") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:20:09.289500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.289474 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:20:09.296376 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.296334 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-web-config" (OuterVolumeSpecName: "web-config") pod "bca47c51-d406-4beb-ad90-33f9519a806f" (UID: "bca47c51-d406-4beb-ad90-33f9519a806f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381144 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-web-config\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381182 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381192 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381203 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmln2\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-kube-api-access-jmln2\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381212 2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-cluster-tls-config\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381220 2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-config-volume\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381229 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381237 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-main-tls\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381246 2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bca47c51-d406-4beb-ad90-33f9519a806f-metrics-client-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381256 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bca47c51-d406-4beb-ad90-33f9519a806f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381655 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381264 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bca47c51-d406-4beb-ad90-33f9519a806f-config-out\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:09.381655 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:09.381271 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bca47c51-d406-4beb-ad90-33f9519a806f-tls-assets\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:10.230004 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.229966 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bca47c51-d406-4beb-ad90-33f9519a806f","Type":"ContainerDied","Data":"7608592e1594fd2f04b5f577a48fb76f6de336aad2d1a65f1ea03b87b40b1971"} Sep 08 15:20:10.230447 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.230050 2578 scope.go:117] "RemoveContainer" containerID="7416998cf6ba7e05201f558372bb749042b5e064f86cf727d73d569932a98bec" Sep 08 15:20:10.230447 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.229976 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.238218 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.238196 2578 scope.go:117] "RemoveContainer" containerID="44a8875260503ad89e1c7589e3f27362509c7bbc3423d1cab403aaec39fdef84" Sep 08 15:20:10.246452 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.246430 2578 scope.go:117] "RemoveContainer" containerID="6874630234c230540099119202f134a5416b403159d0c080dfaf9ef1410d671e" Sep 08 15:20:10.253662 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.253591 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 15:20:10.257577 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.256670 2578 scope.go:117] "RemoveContainer" containerID="641205b53a04fdbab46707375fc9a80318d34b99d749446f788ddf0b2c760915" Sep 08 15:20:10.259519 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.259492 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 15:20:10.266027 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.266006 2578 scope.go:117] "RemoveContainer" containerID="542009a4f76d265b06ee3929f9e0e8eb590b2a94e45cfbd203d7d4b874ab5346" Sep 08 15:20:10.274216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.274196 2578 scope.go:117] "RemoveContainer" containerID="174b39b032d6f812394e5415996536daf503e455dd9d4241248c5d94a9214f89" Sep 08 15:20:10.282426 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.282401 2578 scope.go:117] "RemoveContainer" containerID="ba70f0efaf9a15fcc78ec57b20c37a65cc3b963abb8411501b863ce4c64d54ca" Sep 08 15:20:10.291023 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.290977 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 15:20:10.291325 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291312 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy-web" Sep 08 15:20:10.291373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291328 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy-web" Sep 08 15:20:10.291373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291338 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="config-reloader" Sep 08 15:20:10.291373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291343 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="config-reloader" Sep 08 15:20:10.291373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291349 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="init-config-reloader" Sep 08 15:20:10.291373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291354 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="init-config-reloader" Sep 08 15:20:10.291373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291361 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy" Sep 08 15:20:10.291373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291366 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy" Sep 08 15:20:10.291373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291371 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy-metric" Sep 08 15:20:10.291373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291376 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy-metric" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291383 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="prom-label-proxy" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291387 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="prom-label-proxy" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291399 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="442f5a3e-06f5-4961-8dfa-b62b185ef003" containerName="console" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291403 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="442f5a3e-06f5-4961-8dfa-b62b185ef003" containerName="console" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291409 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="alertmanager" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291414 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="alertmanager" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291466 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy-web" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291473 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="442f5a3e-06f5-4961-8dfa-b62b185ef003" containerName="console" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291479 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="config-reloader" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291484 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy-metric" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291490 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="kube-rbac-proxy" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291495 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="prom-label-proxy" Sep 08 15:20:10.291625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.291501 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" containerName="alertmanager" Sep 08 15:20:10.296652 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.296629 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.299875 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.299846 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 15:20:10.300307 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.300287 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 15:20:10.300419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.300316 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 15:20:10.300717 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.300697 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 15:20:10.300799 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.300701 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 15:20:10.300799 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.300730 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 15:20:10.301096 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.301065 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gv5nn\"" Sep 08 15:20:10.301096 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.301083 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 15:20:10.301278 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.301206 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 15:20:10.308333 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.308307 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 15:20:10.318559 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.318515 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 15:20:10.388197 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388060 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388197 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388142 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388197 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388258 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88f8d6ab-7ca6-429a-bdac-8579387ed04d-config-out\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388297 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88f8d6ab-7ca6-429a-bdac-8579387ed04d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-config-volume\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f8d6ab-7ca6-429a-bdac-8579387ed04d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388419 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/88f8d6ab-7ca6-429a-bdac-8579387ed04d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388696 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-web-config\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388696 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388494 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88f8d6ab-7ca6-429a-bdac-8579387ed04d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388696 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2sbh\" (UniqueName: \"kubernetes.io/projected/88f8d6ab-7ca6-429a-bdac-8579387ed04d-kube-api-access-m2sbh\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.388696 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.388567 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489392 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489392 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88f8d6ab-7ca6-429a-bdac-8579387ed04d-config-out\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489641 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489430 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88f8d6ab-7ca6-429a-bdac-8579387ed04d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489641 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489556 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-config-volume\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489734 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489635 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f8d6ab-7ca6-429a-bdac-8579387ed04d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489734 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/88f8d6ab-7ca6-429a-bdac-8579387ed04d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489734 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489700 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-web-config\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489734 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88f8d6ab-7ca6-429a-bdac-8579387ed04d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2sbh\" (UniqueName: \"kubernetes.io/projected/88f8d6ab-7ca6-429a-bdac-8579387ed04d-kube-api-access-m2sbh\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489841 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489884 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.489936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.489919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.490209 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.490060 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/88f8d6ab-7ca6-429a-bdac-8579387ed04d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.491161 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.490567 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f8d6ab-7ca6-429a-bdac-8579387ed04d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.491161 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.490657 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88f8d6ab-7ca6-429a-bdac-8579387ed04d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.492730 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.492682 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88f8d6ab-7ca6-429a-bdac-8579387ed04d-config-out\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.493194 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.493144 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.493309 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.493234 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-config-volume\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.493447 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.493425 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88f8d6ab-7ca6-429a-bdac-8579387ed04d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.493672 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.493647 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.493817 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.493790 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.493877 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.493863 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.494068 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.494048 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.494798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.494782 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88f8d6ab-7ca6-429a-bdac-8579387ed04d-web-config\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.500388 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.500365 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2sbh\" (UniqueName: \"kubernetes.io/projected/88f8d6ab-7ca6-429a-bdac-8579387ed04d-kube-api-access-m2sbh\") pod \"alertmanager-main-0\" (UID: \"88f8d6ab-7ca6-429a-bdac-8579387ed04d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.607385 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.607345 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 15:20:10.891690 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:10.891660 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 15:20:11.235487 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:11.235452 2578 generic.go:356] "Generic (PLEG): container finished" podID="88f8d6ab-7ca6-429a-bdac-8579387ed04d" containerID="5a0448ee688b15814d6d1ec564fc2bda2536f00726b28d4bb84568f1041496e7" exitCode=0 Sep 08 15:20:11.235919 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:11.235535 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f8d6ab-7ca6-429a-bdac-8579387ed04d","Type":"ContainerDied","Data":"5a0448ee688b15814d6d1ec564fc2bda2536f00726b28d4bb84568f1041496e7"} Sep 08 15:20:11.235919 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:11.235598 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f8d6ab-7ca6-429a-bdac-8579387ed04d","Type":"ContainerStarted","Data":"639367eec788c631224e1b95f10fa6b6319481a3ab4514d4f659f3fe3352d503"} Sep 08 15:20:11.502910 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:11.502868 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bca47c51-d406-4beb-ad90-33f9519a806f" path="/var/lib/kubelet/pods/bca47c51-d406-4beb-ad90-33f9519a806f/volumes" Sep 08 15:20:12.244745 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:12.244702 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f8d6ab-7ca6-429a-bdac-8579387ed04d","Type":"ContainerStarted","Data":"3e3d5e66c7af8731e8fd450b30bdf9f170db2df8cce00f61c107738e18a279da"} Sep 08 15:20:12.245129 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:12.244772 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f8d6ab-7ca6-429a-bdac-8579387ed04d","Type":"ContainerStarted","Data":"5a79888a79bb44497704c107d37d632ae4d46434db289cb46046254bb71f65ed"} Sep 08 15:20:12.245129 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:12.244798 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f8d6ab-7ca6-429a-bdac-8579387ed04d","Type":"ContainerStarted","Data":"99e6e9313f95a69af2828327c5c445ff448ee338ec2b4aac62589e79d68405f2"} Sep 08 15:20:12.245129 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:12.244821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f8d6ab-7ca6-429a-bdac-8579387ed04d","Type":"ContainerStarted","Data":"a8d88990705bc9d8c3148062aeacde66bd20b6b098a639c96b87d77c170f6740"} Sep 08 15:20:12.245129 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:12.244844 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f8d6ab-7ca6-429a-bdac-8579387ed04d","Type":"ContainerStarted","Data":"a0d7314d04a6a103552725f7555a71d471c84224d2b7863358b87a47c3a63c26"} Sep 08 15:20:12.245129 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:12.244912 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f8d6ab-7ca6-429a-bdac-8579387ed04d","Type":"ContainerStarted","Data":"4bfa458895ef6d190a4471898bd2c683d8797a2ec120d6adae029dbce149c35c"} Sep 08 15:20:12.274752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:12.274687 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.259562036 podStartE2EDuration="2.274668211s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.106175ms" isStatefulPod=false podCreationTimestamp="2026-09-08 15:20:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:20:12.273447382 +0000 UTC m=+151.297071839" watchObservedRunningTime="2026-09-08 15:20:12.274668211 +0000 UTC m=+151.298292672" Sep 08 15:20:20.539550 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.539509 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-554d477fb-ms2w6"] Sep 08 15:20:20.543772 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.543730 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.563441 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.563413 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554d477fb-ms2w6"] Sep 08 15:20:20.579023 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.578986 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-service-ca\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.579023 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.579020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxbpk\" (UniqueName: \"kubernetes.io/projected/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-kube-api-access-jxbpk\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.579277 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.579143 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-serving-cert\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.579277 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.579202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-config\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.579387 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.579297 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-oauth-config\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.579387 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.579380 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-trusted-ca-bundle\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.579540 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.579512 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-oauth-serving-cert\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.680320 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.680271 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-trusted-ca-bundle\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.680521 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.680346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-oauth-serving-cert\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.680521 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.680378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-service-ca\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.680521 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.680399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxbpk\" (UniqueName: \"kubernetes.io/projected/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-kube-api-access-jxbpk\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.680521 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.680462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-serving-cert\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.680521 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.680490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-config\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.680766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.680528 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-oauth-config\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.681218 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.681192 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-service-ca\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.681347 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.681230 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-trusted-ca-bundle\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.681347 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.681280 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-config\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.681510 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.681485 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-oauth-serving-cert\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.683225 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.683198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-serving-cert\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.683287 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.683241 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-oauth-config\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.691026 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.690996 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxbpk\" (UniqueName: \"kubernetes.io/projected/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-kube-api-access-jxbpk\") pod \"console-554d477fb-ms2w6\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:20.867708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:20.867604 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:21.146930 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:21.146905 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554d477fb-ms2w6"] Sep 08 15:20:21.149527 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:20:21.149498 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b1c0e2_eab9_4a3d_803e_6f6243cbb598.slice/crio-8235b748e4fd765ac19ca52caf12e1e80049c952d7d2c5525861210b21e21fea WatchSource:0}: Error finding container 8235b748e4fd765ac19ca52caf12e1e80049c952d7d2c5525861210b21e21fea: Status 404 returned error can't find the container with id 8235b748e4fd765ac19ca52caf12e1e80049c952d7d2c5525861210b21e21fea Sep 08 15:20:21.284042 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:21.284011 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554d477fb-ms2w6" event={"ID":"07b1c0e2-eab9-4a3d-803e-6f6243cbb598","Type":"ContainerStarted","Data":"d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346"} Sep 08 15:20:21.284257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:21.284057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554d477fb-ms2w6" event={"ID":"07b1c0e2-eab9-4a3d-803e-6f6243cbb598","Type":"ContainerStarted","Data":"8235b748e4fd765ac19ca52caf12e1e80049c952d7d2c5525861210b21e21fea"} Sep 08 15:20:21.309167 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:21.309100 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-554d477fb-ms2w6" podStartSLOduration=1.3090839029999999 podStartE2EDuration="1.309083903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:20:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:20:21.30876195 +0000 UTC m=+160.332386405" watchObservedRunningTime="2026-09-08 15:20:21.309083903 +0000 UTC m=+160.332708358" Sep 08 15:20:30.867756 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:30.867699 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:30.868300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:30.867799 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:30.872499 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:30.872474 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:31.331199 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:31.331172 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:20:31.391862 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:31.391829 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59db85fd6c-7l975"] Sep 08 15:20:56.412173 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.412092 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59db85fd6c-7l975" podUID="bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" containerName="console" containerID="cri-o://b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7" gracePeriod=15 Sep 08 15:20:56.654418 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.654394 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:20:56.704231 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704202 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvjh7\" (UniqueName: \"kubernetes.io/projected/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-kube-api-access-kvjh7\") pod \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " Sep 08 15:20:56.704408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704247 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-service-ca\") pod \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " Sep 08 15:20:56.704408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704265 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-trusted-ca-bundle\") pod \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " Sep 08 15:20:56.704408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704285 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-oauth-serving-cert\") pod \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " Sep 08 15:20:56.704408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704317 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-serving-cert\") pod \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " Sep 08 15:20:56.704408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704348 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-config\") pod \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " Sep 08 15:20:56.704408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704391 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-oauth-config\") pod \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\" (UID: \"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770\") " Sep 08 15:20:56.704766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704714 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-service-ca" (OuterVolumeSpecName: "service-ca") pod "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" (UID: "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:20:56.704766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704728 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" (UID: "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:20:56.704766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.704724 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" (UID: "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:20:56.705099 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.705001 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-config" (OuterVolumeSpecName: "console-config") pod "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" (UID: "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:20:56.706669 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.706647 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" (UID: "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:20:56.706758 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.706686 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" (UID: "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:20:56.706758 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.706750 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-kube-api-access-kvjh7" (OuterVolumeSpecName: "kube-api-access-kvjh7") pod "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" (UID: "bd28b2da-69b8-4dc0-be9d-e0dd6cf19770"). InnerVolumeSpecName "kube-api-access-kvjh7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:20:56.805071 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.805033 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvjh7\" (UniqueName: \"kubernetes.io/projected/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-kube-api-access-kvjh7\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:56.805071 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.805068 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:56.805071 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.805081 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-trusted-ca-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:56.805332 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.805094 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-oauth-serving-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:56.805332 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.805134 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-serving-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:56.805332 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.805147 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-config\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:56.805332 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:56.805159 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770-console-oauth-config\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:20:57.431215 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.431182 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" containerID="b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7" exitCode=0 Sep 08 15:20:57.431643 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.431220 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59db85fd6c-7l975" event={"ID":"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770","Type":"ContainerDied","Data":"b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7"} Sep 08 15:20:57.431643 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.431229 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59db85fd6c-7l975" Sep 08 15:20:57.431643 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.431265 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59db85fd6c-7l975" event={"ID":"bd28b2da-69b8-4dc0-be9d-e0dd6cf19770","Type":"ContainerDied","Data":"512a10feb3b80d761a7bd4fb8fa1a5ec61733d02616c1814af82af5262c4cff7"} Sep 08 15:20:57.431643 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.431285 2578 scope.go:117] "RemoveContainer" containerID="b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7" Sep 08 15:20:57.442059 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.442034 2578 scope.go:117] "RemoveContainer" containerID="b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7" Sep 08 15:20:57.442415 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:20:57.442393 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7\": container with ID starting with b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7 not found: ID does not exist" containerID="b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7" Sep 08 15:20:57.442523 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.442424 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7"} err="failed to get container status \"b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7\": rpc error: code = NotFound desc = could not find container \"b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7\": container with ID starting with b565221e5dd27c86b593d2060416b4e9db1db5dbc36d349b28ba7dc0493620d7 not found: ID does not exist" Sep 08 15:20:57.471371 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.471339 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59db85fd6c-7l975"] Sep 08 15:20:57.478528 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.478503 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59db85fd6c-7l975"] Sep 08 15:20:57.500993 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:20:57.500959 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" path="/var/lib/kubelet/pods/bd28b2da-69b8-4dc0-be9d-e0dd6cf19770/volumes" Sep 08 15:21:41.201210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.201168 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67"] Sep 08 15:21:41.201813 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.201689 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" containerName="console" Sep 08 15:21:41.201813 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.201710 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" containerName="console" Sep 08 15:21:41.201813 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.201787 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd28b2da-69b8-4dc0-be9d-e0dd6cf19770" containerName="console" Sep 08 15:21:41.205047 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.205024 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.207752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.207733 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 15:21:41.208932 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.208915 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 15:21:41.209028 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.209009 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gd8gf\"" Sep 08 15:21:41.213854 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.213828 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67"] Sep 08 15:21:41.295729 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.295690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.295906 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.295768 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m24n\" (UniqueName: \"kubernetes.io/projected/c056ef6e-236c-490f-a9e4-d8f9075a3441-kube-api-access-6m24n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.295906 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.295799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.397051 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.397013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m24n\" (UniqueName: \"kubernetes.io/projected/c056ef6e-236c-490f-a9e4-d8f9075a3441-kube-api-access-6m24n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.397051 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.397057 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.397277 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.397086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.397443 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.397426 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.397526 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.397506 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.406683 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.406659 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 15:21:41.417479 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.417455 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 15:21:41.427713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.427687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m24n\" (UniqueName: \"kubernetes.io/projected/c056ef6e-236c-490f-a9e4-d8f9075a3441-kube-api-access-6m24n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.517405 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.517312 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gd8gf\"" Sep 08 15:21:41.524581 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.524557 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:21:41.800793 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:41.800760 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67"] Sep 08 15:21:41.803444 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:21:41.803405 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc056ef6e_236c_490f_a9e4_d8f9075a3441.slice/crio-9c26741ccfbd48f961f8dc78ffb2e8e046df1b2b672be3060a70e90f5d4a43f2 WatchSource:0}: Error finding container 9c26741ccfbd48f961f8dc78ffb2e8e046df1b2b672be3060a70e90f5d4a43f2: Status 404 returned error can't find the container with id 9c26741ccfbd48f961f8dc78ffb2e8e046df1b2b672be3060a70e90f5d4a43f2 Sep 08 15:21:42.613170 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:42.613134 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" event={"ID":"c056ef6e-236c-490f-a9e4-d8f9075a3441","Type":"ContainerStarted","Data":"9c26741ccfbd48f961f8dc78ffb2e8e046df1b2b672be3060a70e90f5d4a43f2"} Sep 08 15:21:49.651960 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:49.651917 2578 generic.go:356] "Generic (PLEG): container finished" podID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerID="a8d5189bfe1a244167fec152d3c1028e3005b302eea59a81edfb0b9a8e9621c5" exitCode=0 Sep 08 15:21:49.652453 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:49.652000 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" event={"ID":"c056ef6e-236c-490f-a9e4-d8f9075a3441","Type":"ContainerDied","Data":"a8d5189bfe1a244167fec152d3c1028e3005b302eea59a81edfb0b9a8e9621c5"} Sep 08 15:21:52.667720 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:52.667632 2578 generic.go:356] "Generic (PLEG): container finished" podID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerID="b2f0c1470551988c50f2de977eecadb19593ff404870d0939cc293538a72b37c" exitCode=0 Sep 08 15:21:52.667720 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:52.667687 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" event={"ID":"c056ef6e-236c-490f-a9e4-d8f9075a3441","Type":"ContainerDied","Data":"b2f0c1470551988c50f2de977eecadb19593ff404870d0939cc293538a72b37c"} Sep 08 15:21:59.703660 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:59.703626 2578 generic.go:356] "Generic (PLEG): container finished" podID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerID="bc11879e9b0ec7f34b46f631927dfc315796e80e78212a68c78d45d8c82f5727" exitCode=0 Sep 08 15:21:59.704066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:21:59.703702 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" event={"ID":"c056ef6e-236c-490f-a9e4-d8f9075a3441","Type":"ContainerDied","Data":"bc11879e9b0ec7f34b46f631927dfc315796e80e78212a68c78d45d8c82f5727"} Sep 08 15:22:00.832181 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.832154 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:22:00.845741 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.845707 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-util\") pod \"c056ef6e-236c-490f-a9e4-d8f9075a3441\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " Sep 08 15:22:00.845839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.845753 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m24n\" (UniqueName: \"kubernetes.io/projected/c056ef6e-236c-490f-a9e4-d8f9075a3441-kube-api-access-6m24n\") pod \"c056ef6e-236c-490f-a9e4-d8f9075a3441\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " Sep 08 15:22:00.845875 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.845841 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-bundle\") pod \"c056ef6e-236c-490f-a9e4-d8f9075a3441\" (UID: \"c056ef6e-236c-490f-a9e4-d8f9075a3441\") " Sep 08 15:22:00.846755 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.846715 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-bundle" (OuterVolumeSpecName: "bundle") pod "c056ef6e-236c-490f-a9e4-d8f9075a3441" (UID: "c056ef6e-236c-490f-a9e4-d8f9075a3441"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:22:00.848321 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.848282 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c056ef6e-236c-490f-a9e4-d8f9075a3441-kube-api-access-6m24n" (OuterVolumeSpecName: "kube-api-access-6m24n") pod "c056ef6e-236c-490f-a9e4-d8f9075a3441" (UID: "c056ef6e-236c-490f-a9e4-d8f9075a3441"). InnerVolumeSpecName "kube-api-access-6m24n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:22:00.849761 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.849742 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-util" (OuterVolumeSpecName: "util") pod "c056ef6e-236c-490f-a9e4-d8f9075a3441" (UID: "c056ef6e-236c-490f-a9e4-d8f9075a3441"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:22:00.947202 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.947168 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:00.947202 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.947197 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c056ef6e-236c-490f-a9e4-d8f9075a3441-util\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:00.947202 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:00.947210 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m24n\" (UniqueName: \"kubernetes.io/projected/c056ef6e-236c-490f-a9e4-d8f9075a3441-kube-api-access-6m24n\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:01.713481 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:01.713449 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" Sep 08 15:22:01.713645 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:01.713451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m5s67" event={"ID":"c056ef6e-236c-490f-a9e4-d8f9075a3441","Type":"ContainerDied","Data":"9c26741ccfbd48f961f8dc78ffb2e8e046df1b2b672be3060a70e90f5d4a43f2"} Sep 08 15:22:01.713645 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:01.713567 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c26741ccfbd48f961f8dc78ffb2e8e046df1b2b672be3060a70e90f5d4a43f2" Sep 08 15:22:09.207306 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.207230 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh"] Sep 08 15:22:09.207762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.207554 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerName="util" Sep 08 15:22:09.207762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.207566 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerName="util" Sep 08 15:22:09.207762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.207574 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerName="extract" Sep 08 15:22:09.207762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.207579 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerName="extract" Sep 08 15:22:09.207762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.207586 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerName="pull" Sep 08 15:22:09.207762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.207591 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerName="pull" Sep 08 15:22:09.207762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.207645 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c056ef6e-236c-490f-a9e4-d8f9075a3441" containerName="extract" Sep 08 15:22:09.211711 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.211692 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" Sep 08 15:22:09.214882 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.214859 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 08 15:22:09.214998 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.214859 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-md6fs\"" Sep 08 15:22:09.215217 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.215200 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 08 15:22:09.225237 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.225207 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh"] Sep 08 15:22:09.323952 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.323908 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98445e13-72b5-4f08-a867-fb8f41e7efb8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-j8wzh\" (UID: \"98445e13-72b5-4f08-a867-fb8f41e7efb8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" Sep 08 15:22:09.323952 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.323948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk92x\" (UniqueName: \"kubernetes.io/projected/98445e13-72b5-4f08-a867-fb8f41e7efb8-kube-api-access-hk92x\") pod \"cert-manager-operator-controller-manager-55c444bf86-j8wzh\" (UID: \"98445e13-72b5-4f08-a867-fb8f41e7efb8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" Sep 08 15:22:09.424610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.424573 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98445e13-72b5-4f08-a867-fb8f41e7efb8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-j8wzh\" (UID: \"98445e13-72b5-4f08-a867-fb8f41e7efb8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" Sep 08 15:22:09.424733 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.424621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk92x\" (UniqueName: \"kubernetes.io/projected/98445e13-72b5-4f08-a867-fb8f41e7efb8-kube-api-access-hk92x\") pod \"cert-manager-operator-controller-manager-55c444bf86-j8wzh\" (UID: \"98445e13-72b5-4f08-a867-fb8f41e7efb8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" Sep 08 15:22:09.424959 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.424940 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98445e13-72b5-4f08-a867-fb8f41e7efb8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-j8wzh\" (UID: \"98445e13-72b5-4f08-a867-fb8f41e7efb8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" Sep 08 15:22:09.446368 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.446324 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk92x\" (UniqueName: \"kubernetes.io/projected/98445e13-72b5-4f08-a867-fb8f41e7efb8-kube-api-access-hk92x\") pod \"cert-manager-operator-controller-manager-55c444bf86-j8wzh\" (UID: \"98445e13-72b5-4f08-a867-fb8f41e7efb8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" Sep 08 15:22:09.521578 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.521477 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" Sep 08 15:22:09.801824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:09.801801 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh"] Sep 08 15:22:09.804415 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:22:09.804388 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98445e13_72b5_4f08_a867_fb8f41e7efb8.slice/crio-d93c25637b0a17a5472ddde53bca429ccadb78ae407eb54eef41e36630c717ee WatchSource:0}: Error finding container d93c25637b0a17a5472ddde53bca429ccadb78ae407eb54eef41e36630c717ee: Status 404 returned error can't find the container with id d93c25637b0a17a5472ddde53bca429ccadb78ae407eb54eef41e36630c717ee Sep 08 15:22:10.751069 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:10.751036 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" event={"ID":"98445e13-72b5-4f08-a867-fb8f41e7efb8","Type":"ContainerStarted","Data":"d93c25637b0a17a5472ddde53bca429ccadb78ae407eb54eef41e36630c717ee"} Sep 08 15:22:13.766174 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.766133 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" event={"ID":"98445e13-72b5-4f08-a867-fb8f41e7efb8","Type":"ContainerStarted","Data":"4a5311f2e5e07f563d8e667fb75f5916cbb00be1335c25dd1fea273667e919e7"} Sep 08 15:22:13.790718 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.790657 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j8wzh" podStartSLOduration=1.728064389 podStartE2EDuration="4.790631028s" totalImagesPullingTime="3.062566639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:13.78822227 +0000 UTC m=+272.811846727" watchObservedRunningTime="2026-09-08 15:22:13.790631028 +0000 UTC m=+272.814255488" Sep 08 15:22:13.793072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.793046 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z4b2m"] Sep 08 15:22:13.796384 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.796365 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:13.799233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.799211 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 08 15:22:13.799340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.799235 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-4lkcp\"" Sep 08 15:22:13.799488 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.799469 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 08 15:22:13.806795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.806770 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z4b2m"] Sep 08 15:22:13.862131 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.862066 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c54kq\" (UniqueName: \"kubernetes.io/projected/addee83c-8c71-4de1-9ff5-e54ee03a0e49-kube-api-access-c54kq\") pod \"cert-manager-webhook-6486ffd997-z4b2m\" (UID: \"addee83c-8c71-4de1-9ff5-e54ee03a0e49\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:13.862131 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.862121 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/addee83c-8c71-4de1-9ff5-e54ee03a0e49-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z4b2m\" (UID: \"addee83c-8c71-4de1-9ff5-e54ee03a0e49\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:13.962734 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.962692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c54kq\" (UniqueName: \"kubernetes.io/projected/addee83c-8c71-4de1-9ff5-e54ee03a0e49-kube-api-access-c54kq\") pod \"cert-manager-webhook-6486ffd997-z4b2m\" (UID: \"addee83c-8c71-4de1-9ff5-e54ee03a0e49\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:13.962734 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.962736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/addee83c-8c71-4de1-9ff5-e54ee03a0e49-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z4b2m\" (UID: \"addee83c-8c71-4de1-9ff5-e54ee03a0e49\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:13.971499 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.971472 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/addee83c-8c71-4de1-9ff5-e54ee03a0e49-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z4b2m\" (UID: \"addee83c-8c71-4de1-9ff5-e54ee03a0e49\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:13.971601 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:13.971577 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c54kq\" (UniqueName: \"kubernetes.io/projected/addee83c-8c71-4de1-9ff5-e54ee03a0e49-kube-api-access-c54kq\") pod \"cert-manager-webhook-6486ffd997-z4b2m\" (UID: \"addee83c-8c71-4de1-9ff5-e54ee03a0e49\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:14.114634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.114538 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:14.381549 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.381465 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28"] Sep 08 15:22:14.386023 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.386006 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.388531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.388509 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gd8gf\"" Sep 08 15:22:14.388644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.388618 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 15:22:14.389610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.389591 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 15:22:14.403199 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.403170 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28"] Sep 08 15:22:14.406326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.406303 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z4b2m"] Sep 08 15:22:14.409293 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:22:14.409270 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaddee83c_8c71_4de1_9ff5_e54ee03a0e49.slice/crio-1e06099c10c3c1b650b238a411baf9cfe2c05ce6499ec3ea03360cc6a43b4cc0 WatchSource:0}: Error finding container 1e06099c10c3c1b650b238a411baf9cfe2c05ce6499ec3ea03360cc6a43b4cc0: Status 404 returned error can't find the container with id 1e06099c10c3c1b650b238a411baf9cfe2c05ce6499ec3ea03360cc6a43b4cc0 Sep 08 15:22:14.469044 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.467270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.469044 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.467321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd7qw\" (UniqueName: \"kubernetes.io/projected/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-kube-api-access-fd7qw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.469044 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.467365 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.567856 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.567820 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.568040 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.567861 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd7qw\" (UniqueName: \"kubernetes.io/projected/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-kube-api-access-fd7qw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.568040 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.567904 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.568341 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.568320 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.568388 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.568328 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.590069 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.590043 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd7qw\" (UniqueName: \"kubernetes.io/projected/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-kube-api-access-fd7qw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.695480 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.695397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:14.771929 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.771894 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" event={"ID":"addee83c-8c71-4de1-9ff5-e54ee03a0e49","Type":"ContainerStarted","Data":"1e06099c10c3c1b650b238a411baf9cfe2c05ce6499ec3ea03360cc6a43b4cc0"} Sep 08 15:22:14.978626 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:14.978597 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28"] Sep 08 15:22:14.981343 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:22:14.981313 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4ffcb9_d230_4be7_b9d5_93d33cb391ea.slice/crio-2c38373afbf3e704b675228bb03045e3edc7709eda3ee4db77cfaf2cd4c68dae WatchSource:0}: Error finding container 2c38373afbf3e704b675228bb03045e3edc7709eda3ee4db77cfaf2cd4c68dae: Status 404 returned error can't find the container with id 2c38373afbf3e704b675228bb03045e3edc7709eda3ee4db77cfaf2cd4c68dae Sep 08 15:22:15.783411 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:15.783369 2578 generic.go:356] "Generic (PLEG): container finished" podID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerID="4ee130464a2127295c0bb4c3c14083d9dcf97d0bbe3e7f4e0efa7e66149775fd" exitCode=0 Sep 08 15:22:15.783812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:15.783451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" event={"ID":"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea","Type":"ContainerDied","Data":"4ee130464a2127295c0bb4c3c14083d9dcf97d0bbe3e7f4e0efa7e66149775fd"} Sep 08 15:22:15.783812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:15.783525 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" event={"ID":"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea","Type":"ContainerStarted","Data":"2c38373afbf3e704b675228bb03045e3edc7709eda3ee4db77cfaf2cd4c68dae"} Sep 08 15:22:16.098714 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.098611 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb"] Sep 08 15:22:16.102294 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.102260 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" Sep 08 15:22:16.106085 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.106049 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-bdd6w\"" Sep 08 15:22:16.114396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.114365 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb"] Sep 08 15:22:16.186492 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.186457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvf49\" (UniqueName: \"kubernetes.io/projected/190bdac2-b8a2-4275-be00-b4eeb117526d-kube-api-access-lvf49\") pod \"cert-manager-cainjector-559cdd8c87-hl8xb\" (UID: \"190bdac2-b8a2-4275-be00-b4eeb117526d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" Sep 08 15:22:16.186688 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.186510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/190bdac2-b8a2-4275-be00-b4eeb117526d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hl8xb\" (UID: \"190bdac2-b8a2-4275-be00-b4eeb117526d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" Sep 08 15:22:16.287052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.287013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/190bdac2-b8a2-4275-be00-b4eeb117526d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hl8xb\" (UID: \"190bdac2-b8a2-4275-be00-b4eeb117526d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" Sep 08 15:22:16.287272 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.287165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvf49\" (UniqueName: \"kubernetes.io/projected/190bdac2-b8a2-4275-be00-b4eeb117526d-kube-api-access-lvf49\") pod \"cert-manager-cainjector-559cdd8c87-hl8xb\" (UID: \"190bdac2-b8a2-4275-be00-b4eeb117526d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" Sep 08 15:22:16.297233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.297197 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvf49\" (UniqueName: \"kubernetes.io/projected/190bdac2-b8a2-4275-be00-b4eeb117526d-kube-api-access-lvf49\") pod \"cert-manager-cainjector-559cdd8c87-hl8xb\" (UID: \"190bdac2-b8a2-4275-be00-b4eeb117526d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" Sep 08 15:22:16.297383 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.297264 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/190bdac2-b8a2-4275-be00-b4eeb117526d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hl8xb\" (UID: \"190bdac2-b8a2-4275-be00-b4eeb117526d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" Sep 08 15:22:16.417146 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.416936 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" Sep 08 15:22:16.711861 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.711828 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb"] Sep 08 15:22:16.714267 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:22:16.714225 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod190bdac2_b8a2_4275_be00_b4eeb117526d.slice/crio-86444b58ce75bb0f36366b6d44d87a692b7e88b84a53adddfa3e69e864ba863d WatchSource:0}: Error finding container 86444b58ce75bb0f36366b6d44d87a692b7e88b84a53adddfa3e69e864ba863d: Status 404 returned error can't find the container with id 86444b58ce75bb0f36366b6d44d87a692b7e88b84a53adddfa3e69e864ba863d Sep 08 15:22:16.793316 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:16.793279 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" event={"ID":"190bdac2-b8a2-4275-be00-b4eeb117526d","Type":"ContainerStarted","Data":"86444b58ce75bb0f36366b6d44d87a692b7e88b84a53adddfa3e69e864ba863d"} Sep 08 15:22:18.805759 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:18.805724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" event={"ID":"addee83c-8c71-4de1-9ff5-e54ee03a0e49","Type":"ContainerStarted","Data":"561122f0748a75cc22e5b46badbddd3f7e7e73a2948bf9d04beaf37eb24c0f79"} Sep 08 15:22:18.806232 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:18.805793 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:18.808204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:18.808165 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" event={"ID":"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea","Type":"ContainerStarted","Data":"debee687d77e6b0da05e965f35f538a6fc61e948dd3b5cfb864dc0ea003be957"} Sep 08 15:22:18.810450 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:18.810424 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" event={"ID":"190bdac2-b8a2-4275-be00-b4eeb117526d","Type":"ContainerStarted","Data":"145c2f910b9735e3c51c4985bb2e69ce064725f9333603e6a7fe045d21875db9"} Sep 08 15:22:18.843342 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:18.843283 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" podStartSLOduration=1.5988629840000002 podStartE2EDuration="5.843262762s" totalImagesPullingTime="4.244399778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:18.842155819 +0000 UTC m=+277.865780265" watchObservedRunningTime="2026-09-08 15:22:18.843262762 +0000 UTC m=+277.866887217" Sep 08 15:22:18.917756 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:18.917646 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl8xb" podStartSLOduration=0.977353962 podStartE2EDuration="2.917627042s" totalImagesPullingTime="1.94027308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:18.88547483 +0000 UTC m=+277.909099287" watchObservedRunningTime="2026-09-08 15:22:18.917627042 +0000 UTC m=+277.941251504" Sep 08 15:22:19.817986 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:19.817951 2578 generic.go:356] "Generic (PLEG): container finished" podID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerID="debee687d77e6b0da05e965f35f538a6fc61e948dd3b5cfb864dc0ea003be957" exitCode=0 Sep 08 15:22:19.818507 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:19.818040 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" event={"ID":"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea","Type":"ContainerDied","Data":"debee687d77e6b0da05e965f35f538a6fc61e948dd3b5cfb864dc0ea003be957"} Sep 08 15:22:20.825442 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:20.825402 2578 generic.go:356] "Generic (PLEG): container finished" podID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerID="14c0596659b42b8afe8b7fd619d3769a9e96cedb4366f5b30047a211af655df1" exitCode=0 Sep 08 15:22:20.825812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:20.825459 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" event={"ID":"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea","Type":"ContainerDied","Data":"14c0596659b42b8afe8b7fd619d3769a9e96cedb4366f5b30047a211af655df1"} Sep 08 15:22:21.968714 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:21.968688 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:22.040568 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.040534 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd7qw\" (UniqueName: \"kubernetes.io/projected/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-kube-api-access-fd7qw\") pod \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " Sep 08 15:22:22.040730 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.040598 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-bundle\") pod \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " Sep 08 15:22:22.040730 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.040631 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-util\") pod \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\" (UID: \"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea\") " Sep 08 15:22:22.041022 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.040990 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-bundle" (OuterVolumeSpecName: "bundle") pod "5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" (UID: "5f4ffcb9-d230-4be7-b9d5-93d33cb391ea"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:22:22.042763 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.042740 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-kube-api-access-fd7qw" (OuterVolumeSpecName: "kube-api-access-fd7qw") pod "5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" (UID: "5f4ffcb9-d230-4be7-b9d5-93d33cb391ea"). InnerVolumeSpecName "kube-api-access-fd7qw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:22:22.045639 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.045605 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-util" (OuterVolumeSpecName: "util") pod "5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" (UID: "5f4ffcb9-d230-4be7-b9d5-93d33cb391ea"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:22:22.141384 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.141296 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd7qw\" (UniqueName: \"kubernetes.io/projected/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-kube-api-access-fd7qw\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:22.141384 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.141334 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:22.141384 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.141350 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f4ffcb9-d230-4be7-b9d5-93d33cb391ea-util\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:22.836599 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.836560 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" event={"ID":"5f4ffcb9-d230-4be7-b9d5-93d33cb391ea","Type":"ContainerDied","Data":"2c38373afbf3e704b675228bb03045e3edc7709eda3ee4db77cfaf2cd4c68dae"} Sep 08 15:22:22.836737 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.836629 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c38373afbf3e704b675228bb03045e3edc7709eda3ee4db77cfaf2cd4c68dae" Sep 08 15:22:22.836737 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:22.836634 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmcd28" Sep 08 15:22:24.821169 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:24.821138 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z4b2m" Sep 08 15:22:26.512511 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.512468 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lbm9h"] Sep 08 15:22:26.512883 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.512846 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerName="util" Sep 08 15:22:26.512883 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.512860 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerName="util" Sep 08 15:22:26.512997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.512886 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerName="pull" Sep 08 15:22:26.512997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.512893 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerName="pull" Sep 08 15:22:26.512997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.512906 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerName="extract" Sep 08 15:22:26.512997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.512914 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerName="extract" Sep 08 15:22:26.512997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.512983 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f4ffcb9-d230-4be7-b9d5-93d33cb391ea" containerName="extract" Sep 08 15:22:26.516895 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.516877 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lbm9h" Sep 08 15:22:26.520082 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.520059 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7b9wc\"" Sep 08 15:22:26.534466 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.534442 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lbm9h"] Sep 08 15:22:26.678346 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.678288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw577\" (UniqueName: \"kubernetes.io/projected/4d2e67e0-056f-4f27-8a18-14aa4abe2733-kube-api-access-tw577\") pod \"cert-manager-55b4d954d5-lbm9h\" (UID: \"4d2e67e0-056f-4f27-8a18-14aa4abe2733\") " pod="cert-manager/cert-manager-55b4d954d5-lbm9h" Sep 08 15:22:26.678549 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.678378 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d2e67e0-056f-4f27-8a18-14aa4abe2733-bound-sa-token\") pod \"cert-manager-55b4d954d5-lbm9h\" (UID: \"4d2e67e0-056f-4f27-8a18-14aa4abe2733\") " pod="cert-manager/cert-manager-55b4d954d5-lbm9h" Sep 08 15:22:26.779507 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.779400 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw577\" (UniqueName: \"kubernetes.io/projected/4d2e67e0-056f-4f27-8a18-14aa4abe2733-kube-api-access-tw577\") pod \"cert-manager-55b4d954d5-lbm9h\" (UID: \"4d2e67e0-056f-4f27-8a18-14aa4abe2733\") " pod="cert-manager/cert-manager-55b4d954d5-lbm9h" Sep 08 15:22:26.779507 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.779470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d2e67e0-056f-4f27-8a18-14aa4abe2733-bound-sa-token\") pod \"cert-manager-55b4d954d5-lbm9h\" (UID: \"4d2e67e0-056f-4f27-8a18-14aa4abe2733\") " pod="cert-manager/cert-manager-55b4d954d5-lbm9h" Sep 08 15:22:26.798129 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.798078 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d2e67e0-056f-4f27-8a18-14aa4abe2733-bound-sa-token\") pod \"cert-manager-55b4d954d5-lbm9h\" (UID: \"4d2e67e0-056f-4f27-8a18-14aa4abe2733\") " pod="cert-manager/cert-manager-55b4d954d5-lbm9h" Sep 08 15:22:26.798264 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.798149 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw577\" (UniqueName: \"kubernetes.io/projected/4d2e67e0-056f-4f27-8a18-14aa4abe2733-kube-api-access-tw577\") pod \"cert-manager-55b4d954d5-lbm9h\" (UID: \"4d2e67e0-056f-4f27-8a18-14aa4abe2733\") " pod="cert-manager/cert-manager-55b4d954d5-lbm9h" Sep 08 15:22:26.825258 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:26.825222 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lbm9h" Sep 08 15:22:27.103321 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:27.103293 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lbm9h"] Sep 08 15:22:27.105896 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:22:27.105866 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d2e67e0_056f_4f27_8a18_14aa4abe2733.slice/crio-2b7c04f50f06aaea0f6117cf6390229de08b28719b41c84699db88016aea7f12 WatchSource:0}: Error finding container 2b7c04f50f06aaea0f6117cf6390229de08b28719b41c84699db88016aea7f12: Status 404 returned error can't find the container with id 2b7c04f50f06aaea0f6117cf6390229de08b28719b41c84699db88016aea7f12 Sep 08 15:22:27.863637 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:27.863583 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lbm9h" event={"ID":"4d2e67e0-056f-4f27-8a18-14aa4abe2733","Type":"ContainerStarted","Data":"06534cd5e1bdbe94e27ddd4371040b8ce3cce23d1c9d8151abe8160853bd616a"} Sep 08 15:22:27.864071 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:27.863649 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lbm9h" event={"ID":"4d2e67e0-056f-4f27-8a18-14aa4abe2733","Type":"ContainerStarted","Data":"2b7c04f50f06aaea0f6117cf6390229de08b28719b41c84699db88016aea7f12"} Sep 08 15:22:33.324956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.324903 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lbm9h" podStartSLOduration=7.324886382 podStartE2EDuration="7.324886382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:27.899764447 +0000 UTC m=+286.923388903" watchObservedRunningTime="2026-09-08 15:22:33.324886382 +0000 UTC m=+292.348510838" Sep 08 15:22:33.326101 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.326079 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg"] Sep 08 15:22:33.329520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.329505 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.333299 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.333276 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 15:22:33.333419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.333352 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 15:22:33.334585 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.334568 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gd8gf\"" Sep 08 15:22:33.340443 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.340420 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg"] Sep 08 15:22:33.440465 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.440429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqns4\" (UniqueName: \"kubernetes.io/projected/f15671cb-28e0-44ca-8a08-d2ab60377355-kube-api-access-mqns4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.440465 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.440477 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.440698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.440521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.541256 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.541219 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.541437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.541311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqns4\" (UniqueName: \"kubernetes.io/projected/f15671cb-28e0-44ca-8a08-d2ab60377355-kube-api-access-mqns4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.541437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.541337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.541624 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.541603 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.541667 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.541629 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.551462 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.551431 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqns4\" (UniqueName: \"kubernetes.io/projected/f15671cb-28e0-44ca-8a08-d2ab60377355-kube-api-access-mqns4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.639961 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.639870 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:33.919499 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:33.919472 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg"] Sep 08 15:22:33.921569 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:22:33.921539 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15671cb_28e0_44ca_8a08_d2ab60377355.slice/crio-9d686e8802b644cf55105714c2b99dedf30a8efc2857b568a128ef1e1bc86df9 WatchSource:0}: Error finding container 9d686e8802b644cf55105714c2b99dedf30a8efc2857b568a128ef1e1bc86df9: Status 404 returned error can't find the container with id 9d686e8802b644cf55105714c2b99dedf30a8efc2857b568a128ef1e1bc86df9 Sep 08 15:22:34.897822 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:34.897730 2578 generic.go:356] "Generic (PLEG): container finished" podID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerID="0bc064eb99ae8adffff493487af550159e320d4c25004136866e70e1dd098933" exitCode=0 Sep 08 15:22:34.898196 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:34.897830 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" event={"ID":"f15671cb-28e0-44ca-8a08-d2ab60377355","Type":"ContainerDied","Data":"0bc064eb99ae8adffff493487af550159e320d4c25004136866e70e1dd098933"} Sep 08 15:22:34.898196 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:34.897891 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" event={"ID":"f15671cb-28e0-44ca-8a08-d2ab60377355","Type":"ContainerStarted","Data":"9d686e8802b644cf55105714c2b99dedf30a8efc2857b568a128ef1e1bc86df9"} Sep 08 15:22:35.907175 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:35.907064 2578 generic.go:356] "Generic (PLEG): container finished" podID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerID="dbb1faf133bf10ab827dfb8ee506002f277e4697e8d54ac9a9baf6b57968b9d3" exitCode=0 Sep 08 15:22:35.907175 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:35.907147 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" event={"ID":"f15671cb-28e0-44ca-8a08-d2ab60377355","Type":"ContainerDied","Data":"dbb1faf133bf10ab827dfb8ee506002f277e4697e8d54ac9a9baf6b57968b9d3"} Sep 08 15:22:36.914015 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:36.913980 2578 generic.go:356] "Generic (PLEG): container finished" podID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerID="13b2eb647540a8fb98845c3d0fd88520af2034ad5cb2d790b984a03293ecea5f" exitCode=0 Sep 08 15:22:36.914395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:36.914029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" event={"ID":"f15671cb-28e0-44ca-8a08-d2ab60377355","Type":"ContainerDied","Data":"13b2eb647540a8fb98845c3d0fd88520af2034ad5cb2d790b984a03293ecea5f"} Sep 08 15:22:38.043972 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.043946 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:38.186427 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.186342 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-util\") pod \"f15671cb-28e0-44ca-8a08-d2ab60377355\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " Sep 08 15:22:38.186427 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.186378 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-bundle\") pod \"f15671cb-28e0-44ca-8a08-d2ab60377355\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " Sep 08 15:22:38.186427 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.186422 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqns4\" (UniqueName: \"kubernetes.io/projected/f15671cb-28e0-44ca-8a08-d2ab60377355-kube-api-access-mqns4\") pod \"f15671cb-28e0-44ca-8a08-d2ab60377355\" (UID: \"f15671cb-28e0-44ca-8a08-d2ab60377355\") " Sep 08 15:22:38.187161 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.187078 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-bundle" (OuterVolumeSpecName: "bundle") pod "f15671cb-28e0-44ca-8a08-d2ab60377355" (UID: "f15671cb-28e0-44ca-8a08-d2ab60377355"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:22:38.188626 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.188606 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15671cb-28e0-44ca-8a08-d2ab60377355-kube-api-access-mqns4" (OuterVolumeSpecName: "kube-api-access-mqns4") pod "f15671cb-28e0-44ca-8a08-d2ab60377355" (UID: "f15671cb-28e0-44ca-8a08-d2ab60377355"). InnerVolumeSpecName "kube-api-access-mqns4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:22:38.191925 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.191891 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-util" (OuterVolumeSpecName: "util") pod "f15671cb-28e0-44ca-8a08-d2ab60377355" (UID: "f15671cb-28e0-44ca-8a08-d2ab60377355"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:22:38.287829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.287788 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqns4\" (UniqueName: \"kubernetes.io/projected/f15671cb-28e0-44ca-8a08-d2ab60377355-kube-api-access-mqns4\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:38.287829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.287822 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-util\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:38.287829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.287831 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f15671cb-28e0-44ca-8a08-d2ab60377355-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:38.925768 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.925729 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" event={"ID":"f15671cb-28e0-44ca-8a08-d2ab60377355","Type":"ContainerDied","Data":"9d686e8802b644cf55105714c2b99dedf30a8efc2857b568a128ef1e1bc86df9"} Sep 08 15:22:38.925768 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.925768 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr6rgg" Sep 08 15:22:38.925963 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:38.925779 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d686e8802b644cf55105714c2b99dedf30a8efc2857b568a128ef1e1bc86df9" Sep 08 15:22:41.421619 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:41.421594 2578 kubelet.go:1648] "Image garbage collection succeeded" Sep 08 15:22:50.454987 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.454945 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq"] Sep 08 15:22:50.457537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.455339 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerName="util" Sep 08 15:22:50.457537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.455351 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerName="util" Sep 08 15:22:50.457537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.455368 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerName="pull" Sep 08 15:22:50.457537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.455376 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerName="pull" Sep 08 15:22:50.457537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.455391 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerName="extract" Sep 08 15:22:50.457537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.455397 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerName="extract" Sep 08 15:22:50.457537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.455458 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f15671cb-28e0-44ca-8a08-d2ab60377355" containerName="extract" Sep 08 15:22:50.458751 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.458728 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.463387 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.463361 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 15:22:50.464507 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.464480 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 15:22:50.464622 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.464489 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gd8gf\"" Sep 08 15:22:50.484554 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.484513 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.484722 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.484566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp2ct\" (UniqueName: \"kubernetes.io/projected/24252564-beee-4e1c-9f56-afb93cd096f4-kube-api-access-wp2ct\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.484722 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.484651 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.495158 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.495099 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq"] Sep 08 15:22:50.499726 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.499696 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f"] Sep 08 15:22:50.503050 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.503029 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.507462 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.507437 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wv984\"" Sep 08 15:22:50.507644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.507631 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 08 15:22:50.507706 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.507694 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 08 15:22:50.508146 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.508131 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 08 15:22:50.508697 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.508681 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 08 15:22:50.515379 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.515360 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 08 15:22:50.529215 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.529180 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f"] Sep 08 15:22:50.585364 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.585334 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.585562 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.585524 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b080897-bb15-4de0-b1e1-bb96a0547cf3-metrics-cert\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.585637 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.585578 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b080897-bb15-4de0-b1e1-bb96a0547cf3-manager-config\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.585695 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.585637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w24zh\" (UniqueName: \"kubernetes.io/projected/9b080897-bb15-4de0-b1e1-bb96a0547cf3-kube-api-access-w24zh\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.585751 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.585724 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.585845 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.585807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b080897-bb15-4de0-b1e1-bb96a0547cf3-cert\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.585970 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.585865 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.586099 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.585991 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp2ct\" (UniqueName: \"kubernetes.io/projected/24252564-beee-4e1c-9f56-afb93cd096f4-kube-api-access-wp2ct\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.586251 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.586230 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.602215 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.602184 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp2ct\" (UniqueName: \"kubernetes.io/projected/24252564-beee-4e1c-9f56-afb93cd096f4-kube-api-access-wp2ct\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.640947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.640914 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs"] Sep 08 15:22:50.644399 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.644383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.648646 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.648621 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4lc5s\"" Sep 08 15:22:50.649098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.649078 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 08 15:22:50.651857 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.651840 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 08 15:22:50.654277 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.654255 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 08 15:22:50.658404 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.658388 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 08 15:22:50.674074 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.674045 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs"] Sep 08 15:22:50.687463 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.687436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b080897-bb15-4de0-b1e1-bb96a0547cf3-metrics-cert\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.687604 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.687474 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1be5ebad-b770-4552-aeac-66c2b640898e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.687604 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.687496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b080897-bb15-4de0-b1e1-bb96a0547cf3-manager-config\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.687604 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.687534 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1be5ebad-b770-4552-aeac-66c2b640898e-manifests\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.687604 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.687557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w24zh\" (UniqueName: \"kubernetes.io/projected/9b080897-bb15-4de0-b1e1-bb96a0547cf3-kube-api-access-w24zh\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.687746 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.687615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b080897-bb15-4de0-b1e1-bb96a0547cf3-cert\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.687746 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.687632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1be5ebad-b770-4552-aeac-66c2b640898e-tmp\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.687746 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.687665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1be5ebad-b770-4552-aeac-66c2b640898e-webhook-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.687857 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.687766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br9k2\" (UniqueName: \"kubernetes.io/projected/1be5ebad-b770-4552-aeac-66c2b640898e-kube-api-access-br9k2\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.688260 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.688206 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b080897-bb15-4de0-b1e1-bb96a0547cf3-manager-config\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.690073 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.690053 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b080897-bb15-4de0-b1e1-bb96a0547cf3-cert\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.690073 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.690068 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b080897-bb15-4de0-b1e1-bb96a0547cf3-metrics-cert\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.702509 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.702484 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w24zh\" (UniqueName: \"kubernetes.io/projected/9b080897-bb15-4de0-b1e1-bb96a0547cf3-kube-api-access-w24zh\") pod \"lws-controller-manager-5bc9d6d9f9-4vg4f\" (UID: \"9b080897-bb15-4de0-b1e1-bb96a0547cf3\") " pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.767866 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.767776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:50.788412 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.788385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br9k2\" (UniqueName: \"kubernetes.io/projected/1be5ebad-b770-4552-aeac-66c2b640898e-kube-api-access-br9k2\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.788537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.788429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1be5ebad-b770-4552-aeac-66c2b640898e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.788537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.788454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1be5ebad-b770-4552-aeac-66c2b640898e-manifests\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.788537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.788501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1be5ebad-b770-4552-aeac-66c2b640898e-tmp\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.788537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.788531 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1be5ebad-b770-4552-aeac-66c2b640898e-webhook-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.788943 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.788919 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1be5ebad-b770-4552-aeac-66c2b640898e-manifests\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.790891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.790868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1be5ebad-b770-4552-aeac-66c2b640898e-tmp\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.791123 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.791094 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1be5ebad-b770-4552-aeac-66c2b640898e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.791599 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.791576 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1be5ebad-b770-4552-aeac-66c2b640898e-webhook-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.807194 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.807131 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br9k2\" (UniqueName: \"kubernetes.io/projected/1be5ebad-b770-4552-aeac-66c2b640898e-kube-api-access-br9k2\") pod \"opendatahub-operator-controller-manager-847d47f6c8-fllhs\" (UID: \"1be5ebad-b770-4552-aeac-66c2b640898e\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:50.812825 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.812797 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:50.954736 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:50.954693 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:51.055900 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:51.055870 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq"] Sep 08 15:22:51.057591 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:22:51.057567 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24252564_beee_4e1c_9f56_afb93cd096f4.slice/crio-2d9ddcf7811b6ead8b9679522ac086a0cef30fbc04c1bf444266b370c597d4ba WatchSource:0}: Error finding container 2d9ddcf7811b6ead8b9679522ac086a0cef30fbc04c1bf444266b370c597d4ba: Status 404 returned error can't find the container with id 2d9ddcf7811b6ead8b9679522ac086a0cef30fbc04c1bf444266b370c597d4ba Sep 08 15:22:51.114789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:51.114764 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f"] Sep 08 15:22:51.116577 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:22:51.116553 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b080897_bb15_4de0_b1e1_bb96a0547cf3.slice/crio-e02865adc7692938eb78ee7145aab83e1e2ceae4b449f22dde9c0c9bef64384e WatchSource:0}: Error finding container e02865adc7692938eb78ee7145aab83e1e2ceae4b449f22dde9c0c9bef64384e: Status 404 returned error can't find the container with id e02865adc7692938eb78ee7145aab83e1e2ceae4b449f22dde9c0c9bef64384e Sep 08 15:22:51.119078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:51.119063 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 15:22:51.243636 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:51.243572 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs"] Sep 08 15:22:51.245873 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:22:51.245839 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be5ebad_b770_4552_aeac_66c2b640898e.slice/crio-ba1a5086fc97fe0784f82e1d18d207e4681ff0b9ba30bb2687cf5e9168f499e4 WatchSource:0}: Error finding container ba1a5086fc97fe0784f82e1d18d207e4681ff0b9ba30bb2687cf5e9168f499e4: Status 404 returned error can't find the container with id ba1a5086fc97fe0784f82e1d18d207e4681ff0b9ba30bb2687cf5e9168f499e4 Sep 08 15:22:52.000620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:52.000518 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" event={"ID":"1be5ebad-b770-4552-aeac-66c2b640898e","Type":"ContainerStarted","Data":"ba1a5086fc97fe0784f82e1d18d207e4681ff0b9ba30bb2687cf5e9168f499e4"} Sep 08 15:22:52.002710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:52.002673 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" event={"ID":"9b080897-bb15-4de0-b1e1-bb96a0547cf3","Type":"ContainerStarted","Data":"e02865adc7692938eb78ee7145aab83e1e2ceae4b449f22dde9c0c9bef64384e"} Sep 08 15:22:52.005557 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:52.005519 2578 generic.go:356] "Generic (PLEG): container finished" podID="24252564-beee-4e1c-9f56-afb93cd096f4" containerID="bee44c74a725f3bf48d90281e6f38a873d121f30a30a5f9ae19c66d5a9d3ded4" exitCode=0 Sep 08 15:22:52.005710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:52.005650 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" event={"ID":"24252564-beee-4e1c-9f56-afb93cd096f4","Type":"ContainerDied","Data":"bee44c74a725f3bf48d90281e6f38a873d121f30a30a5f9ae19c66d5a9d3ded4"} Sep 08 15:22:52.005710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:52.005695 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" event={"ID":"24252564-beee-4e1c-9f56-afb93cd096f4","Type":"ContainerStarted","Data":"2d9ddcf7811b6ead8b9679522ac086a0cef30fbc04c1bf444266b370c597d4ba"} Sep 08 15:22:53.015465 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:53.015364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" event={"ID":"24252564-beee-4e1c-9f56-afb93cd096f4","Type":"ContainerStarted","Data":"a55266a0c7044db3b3c6e407c5a6181386649076cf57a33402a1ae24abe0a1de"} Sep 08 15:22:54.023845 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:54.023799 2578 generic.go:356] "Generic (PLEG): container finished" podID="24252564-beee-4e1c-9f56-afb93cd096f4" containerID="a55266a0c7044db3b3c6e407c5a6181386649076cf57a33402a1ae24abe0a1de" exitCode=0 Sep 08 15:22:54.024334 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:54.023919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" event={"ID":"24252564-beee-4e1c-9f56-afb93cd096f4","Type":"ContainerDied","Data":"a55266a0c7044db3b3c6e407c5a6181386649076cf57a33402a1ae24abe0a1de"} Sep 08 15:22:55.031969 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:55.031932 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" event={"ID":"1be5ebad-b770-4552-aeac-66c2b640898e","Type":"ContainerStarted","Data":"930cf0fcffd9a32c3a22622a2385948c80aaf996094bea20adee37103a62caee"} Sep 08 15:22:55.033640 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:55.033611 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" event={"ID":"9b080897-bb15-4de0-b1e1-bb96a0547cf3","Type":"ContainerStarted","Data":"65c8303ddd3800aea6e2feeeb2151299cb101915c45bca80e779a97a2b5fe56d"} Sep 08 15:22:55.033763 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:55.033706 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:22:55.036147 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:55.036101 2578 generic.go:356] "Generic (PLEG): container finished" podID="24252564-beee-4e1c-9f56-afb93cd096f4" containerID="32cdf5cd4b595e31466e84070a1419824b77014d6f0294804fb740e940e73720" exitCode=0 Sep 08 15:22:55.036261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:55.036181 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" event={"ID":"24252564-beee-4e1c-9f56-afb93cd096f4","Type":"ContainerDied","Data":"32cdf5cd4b595e31466e84070a1419824b77014d6f0294804fb740e940e73720"} Sep 08 15:22:55.083288 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:55.083226 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" podStartSLOduration=1.249762296 podStartE2EDuration="5.083203403s" totalImagesPullingTime="3.833441107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:55.082727056 +0000 UTC m=+314.106351512" watchObservedRunningTime="2026-09-08 15:22:55.083203403 +0000 UTC m=+314.106827859" Sep 08 15:22:56.175139 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.175098 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:56.236840 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.236802 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-util\") pod \"24252564-beee-4e1c-9f56-afb93cd096f4\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " Sep 08 15:22:56.236984 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.236858 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-bundle\") pod \"24252564-beee-4e1c-9f56-afb93cd096f4\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " Sep 08 15:22:56.236984 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.236968 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp2ct\" (UniqueName: \"kubernetes.io/projected/24252564-beee-4e1c-9f56-afb93cd096f4-kube-api-access-wp2ct\") pod \"24252564-beee-4e1c-9f56-afb93cd096f4\" (UID: \"24252564-beee-4e1c-9f56-afb93cd096f4\") " Sep 08 15:22:56.237627 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.237598 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-bundle" (OuterVolumeSpecName: "bundle") pod "24252564-beee-4e1c-9f56-afb93cd096f4" (UID: "24252564-beee-4e1c-9f56-afb93cd096f4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:22:56.239199 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.239164 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24252564-beee-4e1c-9f56-afb93cd096f4-kube-api-access-wp2ct" (OuterVolumeSpecName: "kube-api-access-wp2ct") pod "24252564-beee-4e1c-9f56-afb93cd096f4" (UID: "24252564-beee-4e1c-9f56-afb93cd096f4"). InnerVolumeSpecName "kube-api-access-wp2ct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:22:56.242885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.242866 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-util" (OuterVolumeSpecName: "util") pod "24252564-beee-4e1c-9f56-afb93cd096f4" (UID: "24252564-beee-4e1c-9f56-afb93cd096f4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:22:56.338225 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.338142 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp2ct\" (UniqueName: \"kubernetes.io/projected/24252564-beee-4e1c-9f56-afb93cd096f4-kube-api-access-wp2ct\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:56.338225 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.338172 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-util\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:56.338225 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:56.338185 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24252564-beee-4e1c-9f56-afb93cd096f4-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:22:57.049247 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:57.049214 2578 generic.go:356] "Generic (PLEG): container finished" podID="1be5ebad-b770-4552-aeac-66c2b640898e" containerID="930cf0fcffd9a32c3a22622a2385948c80aaf996094bea20adee37103a62caee" exitCode=0 Sep 08 15:22:57.049432 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:57.049283 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" event={"ID":"1be5ebad-b770-4552-aeac-66c2b640898e","Type":"ContainerDied","Data":"930cf0fcffd9a32c3a22622a2385948c80aaf996094bea20adee37103a62caee"} Sep 08 15:22:57.051336 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:57.051315 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" event={"ID":"24252564-beee-4e1c-9f56-afb93cd096f4","Type":"ContainerDied","Data":"2d9ddcf7811b6ead8b9679522ac086a0cef30fbc04c1bf444266b370c597d4ba"} Sep 08 15:22:57.051442 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:57.051350 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d9ddcf7811b6ead8b9679522ac086a0cef30fbc04c1bf444266b370c597d4ba" Sep 08 15:22:57.051442 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:57.051366 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0gj9vq" Sep 08 15:22:58.060166 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:58.060100 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" event={"ID":"1be5ebad-b770-4552-aeac-66c2b640898e","Type":"ContainerStarted","Data":"730370f99f54be6822eee3df0f56861ffd0f1d0e2a88972809ac40fa6aa754b7"} Sep 08 15:22:58.060599 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:58.060407 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:22:58.086863 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:22:58.086805 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" podStartSLOduration=3.030618371 podStartE2EDuration="8.086786954s" totalImagesPullingTime="4.046073459s" totalInitContainerRuntime="1.010095124s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:58.083995253 +0000 UTC m=+317.107619700" watchObservedRunningTime="2026-09-08 15:22:58.086786954 +0000 UTC m=+317.110411410" Sep 08 15:23:06.045188 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:06.045152 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bc9d6d9f9-4vg4f" Sep 08 15:23:09.069080 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:09.069041 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-fllhs" Sep 08 15:23:20.152153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.150446 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j"] Sep 08 15:23:20.152153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.151343 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24252564-beee-4e1c-9f56-afb93cd096f4" containerName="pull" Sep 08 15:23:20.152153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.151369 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="24252564-beee-4e1c-9f56-afb93cd096f4" containerName="pull" Sep 08 15:23:20.152153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.151392 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24252564-beee-4e1c-9f56-afb93cd096f4" containerName="extract" Sep 08 15:23:20.152153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.151401 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="24252564-beee-4e1c-9f56-afb93cd096f4" containerName="extract" Sep 08 15:23:20.152153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.151416 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24252564-beee-4e1c-9f56-afb93cd096f4" containerName="util" Sep 08 15:23:20.152153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.151430 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="24252564-beee-4e1c-9f56-afb93cd096f4" containerName="util" Sep 08 15:23:20.152153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.151626 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="24252564-beee-4e1c-9f56-afb93cd096f4" containerName="extract" Sep 08 15:23:20.163421 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.163390 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.168431 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.168409 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-54fsn\"" Sep 08 15:23:20.169305 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.169283 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 08 15:23:20.170496 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.170475 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 08 15:23:20.170599 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.170552 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 08 15:23:20.182633 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.182599 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j"] Sep 08 15:23:20.262223 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.262187 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.262395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.262232 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d81c7232-d576-4049-9669-14e3e423fa43-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.262395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.262256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.262395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.262312 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq6c5\" (UniqueName: \"kubernetes.io/projected/d81c7232-d576-4049-9669-14e3e423fa43-kube-api-access-nq6c5\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.262395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.262358 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.262395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.262385 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d81c7232-d576-4049-9669-14e3e423fa43-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.262561 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.262467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.262561 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.262491 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.262561 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.262506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81c7232-d576-4049-9669-14e3e423fa43-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.363626 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.363523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.363626 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.363567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81c7232-d576-4049-9669-14e3e423fa43-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.363856 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.363633 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.363856 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.363668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d81c7232-d576-4049-9669-14e3e423fa43-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.363856 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.363700 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.363856 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.363725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq6c5\" (UniqueName: \"kubernetes.io/projected/d81c7232-d576-4049-9669-14e3e423fa43-kube-api-access-nq6c5\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.363856 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.363763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.363856 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.363787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d81c7232-d576-4049-9669-14e3e423fa43-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.364296 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.363861 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.364296 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.364020 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.364296 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.364083 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.364296 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.364186 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.364296 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.364265 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.364474 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.364453 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d81c7232-d576-4049-9669-14e3e423fa43-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.366506 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.366481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d81c7232-d576-4049-9669-14e3e423fa43-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.366759 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.366741 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d81c7232-d576-4049-9669-14e3e423fa43-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.373883 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.373855 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81c7232-d576-4049-9669-14e3e423fa43-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.374031 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.373984 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq6c5\" (UniqueName: \"kubernetes.io/projected/d81c7232-d576-4049-9669-14e3e423fa43-kube-api-access-nq6c5\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j\" (UID: \"d81c7232-d576-4049-9669-14e3e423fa43\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.473650 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.473615 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:20.759063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:20.759037 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j"] Sep 08 15:23:20.761094 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:23:20.761061 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81c7232_d576_4049_9669_14e3e423fa43.slice/crio-b46947bb5b24bc0cf50a9d32862d103a74f70a838ee9c4e2cc901ab7afd589f9 WatchSource:0}: Error finding container b46947bb5b24bc0cf50a9d32862d103a74f70a838ee9c4e2cc901ab7afd589f9: Status 404 returned error can't find the container with id b46947bb5b24bc0cf50a9d32862d103a74f70a838ee9c4e2cc901ab7afd589f9 Sep 08 15:23:21.177443 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:21.177350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" event={"ID":"d81c7232-d576-4049-9669-14e3e423fa43","Type":"ContainerStarted","Data":"b46947bb5b24bc0cf50a9d32862d103a74f70a838ee9c4e2cc901ab7afd589f9"} Sep 08 15:23:23.615195 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:23.615161 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:23:23.615452 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:23.615232 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:23:23.615452 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:23.615261 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:23:23.921546 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:23.921461 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vfdjb"] Sep 08 15:23:23.925756 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:23.925728 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:23.928610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:23.928582 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 08 15:23:23.936354 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:23.936308 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vfdjb"] Sep 08 15:23:24.098402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.098364 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63fe5d59-d590-4aab-91c3-0883934916b5-tls-certs\") pod \"model-serving-api-54d89bd5fc-vfdjb\" (UID: \"63fe5d59-d590-4aab-91c3-0883934916b5\") " pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:24.098576 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.098407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkwll\" (UniqueName: \"kubernetes.io/projected/63fe5d59-d590-4aab-91c3-0883934916b5-kube-api-access-gkwll\") pod \"model-serving-api-54d89bd5fc-vfdjb\" (UID: \"63fe5d59-d590-4aab-91c3-0883934916b5\") " pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:24.202279 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.202072 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63fe5d59-d590-4aab-91c3-0883934916b5-tls-certs\") pod \"model-serving-api-54d89bd5fc-vfdjb\" (UID: \"63fe5d59-d590-4aab-91c3-0883934916b5\") " pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:24.202279 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.202146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkwll\" (UniqueName: \"kubernetes.io/projected/63fe5d59-d590-4aab-91c3-0883934916b5-kube-api-access-gkwll\") pod \"model-serving-api-54d89bd5fc-vfdjb\" (UID: \"63fe5d59-d590-4aab-91c3-0883934916b5\") " pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:24.202279 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:23:24.202239 2578 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Sep 08 15:23:24.202537 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:23:24.202313 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63fe5d59-d590-4aab-91c3-0883934916b5-tls-certs podName:63fe5d59-d590-4aab-91c3-0883934916b5 nodeName:}" failed. No retries permitted until 2026-09-08 15:23:24.702290915 +0000 UTC m=+343.725915355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/63fe5d59-d590-4aab-91c3-0883934916b5-tls-certs") pod "model-serving-api-54d89bd5fc-vfdjb" (UID: "63fe5d59-d590-4aab-91c3-0883934916b5") : secret "model-serving-api-tls" not found Sep 08 15:23:24.207480 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.207449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" event={"ID":"d81c7232-d576-4049-9669-14e3e423fa43","Type":"ContainerStarted","Data":"515484cc6120b5097df82ab7065cb42c136caed7aedd521505ae19f4a57b8ab2"} Sep 08 15:23:24.213421 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.213397 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkwll\" (UniqueName: \"kubernetes.io/projected/63fe5d59-d590-4aab-91c3-0883934916b5-kube-api-access-gkwll\") pod \"model-serving-api-54d89bd5fc-vfdjb\" (UID: \"63fe5d59-d590-4aab-91c3-0883934916b5\") " pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:24.229229 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.229173 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" podStartSLOduration=1.3773021349999999 podStartE2EDuration="4.229153109s" totalImagesPullingTime="2.851850974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:23:24.228007916 +0000 UTC m=+343.251632373" watchObservedRunningTime="2026-09-08 15:23:24.229153109 +0000 UTC m=+343.252777567" Sep 08 15:23:24.474168 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.474045 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:24.483863 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.483826 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:24.706834 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.706790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63fe5d59-d590-4aab-91c3-0883934916b5-tls-certs\") pod \"model-serving-api-54d89bd5fc-vfdjb\" (UID: \"63fe5d59-d590-4aab-91c3-0883934916b5\") " pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:24.709414 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.709384 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63fe5d59-d590-4aab-91c3-0883934916b5-tls-certs\") pod \"model-serving-api-54d89bd5fc-vfdjb\" (UID: \"63fe5d59-d590-4aab-91c3-0883934916b5\") " pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:24.841098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:24.840988 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:25.119570 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:25.119539 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vfdjb"] Sep 08 15:23:25.121226 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:23:25.121195 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63fe5d59_d590_4aab_91c3_0883934916b5.slice/crio-8ff81845d4ef987c07df78878b8c3ac2c27bde94f78e6b91740bf219a90c882a WatchSource:0}: Error finding container 8ff81845d4ef987c07df78878b8c3ac2c27bde94f78e6b91740bf219a90c882a: Status 404 returned error can't find the container with id 8ff81845d4ef987c07df78878b8c3ac2c27bde94f78e6b91740bf219a90c882a Sep 08 15:23:25.214188 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:25.214147 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" event={"ID":"63fe5d59-d590-4aab-91c3-0883934916b5","Type":"ContainerStarted","Data":"8ff81845d4ef987c07df78878b8c3ac2c27bde94f78e6b91740bf219a90c882a"} Sep 08 15:23:25.214372 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:25.214349 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:25.215436 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:25.215419 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j" Sep 08 15:23:28.232202 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:28.232165 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" event={"ID":"63fe5d59-d590-4aab-91c3-0883934916b5","Type":"ContainerStarted","Data":"45e4b1c03e68bc2c84f264fb349d677eedfb26ba3234ab53a4888ae0ca344c4e"} Sep 08 15:23:28.232624 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:28.232264 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:23:28.253147 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:28.253082 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" podStartSLOduration=2.716674298 podStartE2EDuration="5.253064394s" totalImagesPullingTime="2.536390096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:23:28.250826417 +0000 UTC m=+347.274450874" watchObservedRunningTime="2026-09-08 15:23:28.253064394 +0000 UTC m=+347.276688850" Sep 08 15:23:34.466518 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.466433 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9"] Sep 08 15:23:34.470396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.470367 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:23:34.473273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.473250 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 08 15:23:34.473890 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.473852 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kd65v\"" Sep 08 15:23:34.482557 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.482527 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9"] Sep 08 15:23:34.505811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.505613 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtl97\" (UniqueName: \"kubernetes.io/projected/01884b48-b914-4524-ba96-06c0840aac59-kube-api-access-mtl97\") pod \"kserve-controller-manager-d6f4b65b8-pk6h9\" (UID: \"01884b48-b914-4524-ba96-06c0840aac59\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:23:34.505811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.505683 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01884b48-b914-4524-ba96-06c0840aac59-cert\") pod \"kserve-controller-manager-d6f4b65b8-pk6h9\" (UID: \"01884b48-b914-4524-ba96-06c0840aac59\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:23:34.606734 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.606694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtl97\" (UniqueName: \"kubernetes.io/projected/01884b48-b914-4524-ba96-06c0840aac59-kube-api-access-mtl97\") pod \"kserve-controller-manager-d6f4b65b8-pk6h9\" (UID: \"01884b48-b914-4524-ba96-06c0840aac59\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:23:34.606928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.606748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01884b48-b914-4524-ba96-06c0840aac59-cert\") pod \"kserve-controller-manager-d6f4b65b8-pk6h9\" (UID: \"01884b48-b914-4524-ba96-06c0840aac59\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:23:34.609423 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.609400 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01884b48-b914-4524-ba96-06c0840aac59-cert\") pod \"kserve-controller-manager-d6f4b65b8-pk6h9\" (UID: \"01884b48-b914-4524-ba96-06c0840aac59\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:23:34.617077 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.617056 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtl97\" (UniqueName: \"kubernetes.io/projected/01884b48-b914-4524-ba96-06c0840aac59-kube-api-access-mtl97\") pod \"kserve-controller-manager-d6f4b65b8-pk6h9\" (UID: \"01884b48-b914-4524-ba96-06c0840aac59\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:23:34.784627 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:34.784530 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:23:35.143806 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:35.139967 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9"] Sep 08 15:23:35.274382 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:35.274327 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" event={"ID":"01884b48-b914-4524-ba96-06c0840aac59","Type":"ContainerStarted","Data":"86257f4706f07b9c8a3a50e59910531b6931a6ea718195bc6372eebd2b2f9d1f"} Sep 08 15:23:37.288307 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:37.288269 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" event={"ID":"01884b48-b914-4524-ba96-06c0840aac59","Type":"ContainerStarted","Data":"331688d4a427a1828b8f6e2a6b92d67704ba8c8210b45ae8acf59837357736c8"} Sep 08 15:23:37.288718 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:37.288454 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:23:37.358207 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:37.358155 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" podStartSLOduration=1.300104231 podStartE2EDuration="3.358138223s" totalImagesPullingTime="2.058033992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:23:37.356168412 +0000 UTC m=+356.379792868" watchObservedRunningTime="2026-09-08 15:23:37.358138223 +0000 UTC m=+356.381762676" Sep 08 15:23:39.242828 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:23:39.242800 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vfdjb" Sep 08 15:24:05.781635 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.781586 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66995fbfd7-zf9fn"] Sep 08 15:24:05.785256 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.785233 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:05.800996 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.800963 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66995fbfd7-zf9fn"] Sep 08 15:24:05.901931 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.901875 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b4ddfc9-7e76-4dc3-960b-1905914dd893-console-oauth-config\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:05.901931 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.901941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-trusted-ca-bundle\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:05.902200 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.901976 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-service-ca\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:05.902200 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.901995 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b4ddfc9-7e76-4dc3-960b-1905914dd893-console-serving-cert\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:05.902200 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.902020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-oauth-serving-cert\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:05.902314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.902210 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8pzd\" (UniqueName: \"kubernetes.io/projected/9b4ddfc9-7e76-4dc3-960b-1905914dd893-kube-api-access-f8pzd\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:05.902314 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:05.902250 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-console-config\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.002753 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.002713 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8pzd\" (UniqueName: \"kubernetes.io/projected/9b4ddfc9-7e76-4dc3-960b-1905914dd893-kube-api-access-f8pzd\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.002753 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.002758 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-console-config\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.003032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.002775 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b4ddfc9-7e76-4dc3-960b-1905914dd893-console-oauth-config\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.003032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.002798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-trusted-ca-bundle\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.003032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.002817 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-service-ca\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.003032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.002836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b4ddfc9-7e76-4dc3-960b-1905914dd893-console-serving-cert\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.003032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.002862 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-oauth-serving-cert\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.003671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.003643 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-oauth-serving-cert\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.003827 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.003803 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-trusted-ca-bundle\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.003894 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.003832 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-console-config\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.003894 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.003858 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b4ddfc9-7e76-4dc3-960b-1905914dd893-service-ca\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.005757 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.005726 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b4ddfc9-7e76-4dc3-960b-1905914dd893-console-serving-cert\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.005886 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.005807 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b4ddfc9-7e76-4dc3-960b-1905914dd893-console-oauth-config\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.012034 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.012007 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8pzd\" (UniqueName: \"kubernetes.io/projected/9b4ddfc9-7e76-4dc3-960b-1905914dd893-kube-api-access-f8pzd\") pod \"console-66995fbfd7-zf9fn\" (UID: \"9b4ddfc9-7e76-4dc3-960b-1905914dd893\") " pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.097195 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.097075 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:06.376702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.376675 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66995fbfd7-zf9fn"] Sep 08 15:24:06.379003 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:24:06.378973 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b4ddfc9_7e76_4dc3_960b_1905914dd893.slice/crio-9c7dbbe5d8f807b483a7404ee43705dd994421bf27deea69861dac982240f575 WatchSource:0}: Error finding container 9c7dbbe5d8f807b483a7404ee43705dd994421bf27deea69861dac982240f575: Status 404 returned error can't find the container with id 9c7dbbe5d8f807b483a7404ee43705dd994421bf27deea69861dac982240f575 Sep 08 15:24:06.438617 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:06.438585 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66995fbfd7-zf9fn" event={"ID":"9b4ddfc9-7e76-4dc3-960b-1905914dd893","Type":"ContainerStarted","Data":"9c7dbbe5d8f807b483a7404ee43705dd994421bf27deea69861dac982240f575"} Sep 08 15:24:07.446329 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:07.446291 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66995fbfd7-zf9fn" event={"ID":"9b4ddfc9-7e76-4dc3-960b-1905914dd893","Type":"ContainerStarted","Data":"d48d1511c559c41dde81462cd534e3a17219e99876fa260b2b470f5dd973c693"} Sep 08 15:24:07.468076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:07.468016 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66995fbfd7-zf9fn" podStartSLOduration=2.467998741 podStartE2EDuration="2.467998741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:24:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:24:07.465985423 +0000 UTC m=+386.489609873" watchObservedRunningTime="2026-09-08 15:24:07.467998741 +0000 UTC m=+386.491623196" Sep 08 15:24:08.298913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:08.298880 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pk6h9" Sep 08 15:24:16.097399 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:16.097338 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:16.097999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:16.097425 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:16.102204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:16.102177 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:16.506387 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:16.506355 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66995fbfd7-zf9fn" Sep 08 15:24:16.569901 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:16.569851 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554d477fb-ms2w6"] Sep 08 15:24:28.285126 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.285078 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg"] Sep 08 15:24:28.289239 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.289219 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.292068 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.292041 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 15:24:28.292241 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.292181 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gd8gf\"" Sep 08 15:24:28.293134 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.293098 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 15:24:28.299913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.299873 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg"] Sep 08 15:24:28.383824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.383790 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h"] Sep 08 15:24:28.387539 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.387516 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.397477 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.397451 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h"] Sep 08 15:24:28.401208 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.401180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.401339 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.401239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.401339 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.401321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g27w2\" (UniqueName: \"kubernetes.io/projected/d04eed1a-f8b1-45e2-9701-946e5b59f25f-kube-api-access-g27w2\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.483233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.483194 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh"] Sep 08 15:24:28.487303 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.487281 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.497384 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.497356 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh"] Sep 08 15:24:28.502074 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.502045 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mmj8\" (UniqueName: \"kubernetes.io/projected/3975d31a-b48f-4133-ba5f-72a1d8f11485-kube-api-access-8mmj8\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.502233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.502086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.502233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.502103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.502345 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.502263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.502395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.502345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.502443 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.502402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g27w2\" (UniqueName: \"kubernetes.io/projected/d04eed1a-f8b1-45e2-9701-946e5b59f25f-kube-api-access-g27w2\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.502508 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.502493 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.502613 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.502595 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.511632 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.511604 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g27w2\" (UniqueName: \"kubernetes.io/projected/d04eed1a-f8b1-45e2-9701-946e5b59f25f-kube-api-access-g27w2\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.592222 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.592138 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb"] Sep 08 15:24:28.595982 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.595966 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.601661 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.601639 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:28.603648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.603054 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.603648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.603098 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59vhv\" (UniqueName: \"kubernetes.io/projected/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-kube-api-access-59vhv\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.603648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.603163 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.603648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.603305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.603648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.603376 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mmj8\" (UniqueName: \"kubernetes.io/projected/3975d31a-b48f-4133-ba5f-72a1d8f11485-kube-api-access-8mmj8\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.603648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.603420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.603961 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.603735 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.604584 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.604561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.608163 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.608136 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb"] Sep 08 15:24:28.614466 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.614434 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mmj8\" (UniqueName: \"kubernetes.io/projected/3975d31a-b48f-4133-ba5f-72a1d8f11485-kube-api-access-8mmj8\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.698009 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.697971 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:28.704740 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.704700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2rvd\" (UniqueName: \"kubernetes.io/projected/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-kube-api-access-k2rvd\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.704865 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.704753 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.704915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.704871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.704984 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.704966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.705045 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.704999 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.705045 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.705030 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59vhv\" (UniqueName: \"kubernetes.io/projected/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-kube-api-access-59vhv\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.705379 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.705317 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.705466 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.705418 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.716854 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.716825 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59vhv\" (UniqueName: \"kubernetes.io/projected/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-kube-api-access-59vhv\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.796776 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.796744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:28.806261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.806238 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2rvd\" (UniqueName: \"kubernetes.io/projected/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-kube-api-access-k2rvd\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.806352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.806279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.806352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.806341 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.806666 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.806641 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.806741 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.806686 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.818059 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.818028 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2rvd\" (UniqueName: \"kubernetes.io/projected/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-kube-api-access-k2rvd\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.883695 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.883671 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg"] Sep 08 15:24:28.886694 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:24:28.886663 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd04eed1a_f8b1_45e2_9701_946e5b59f25f.slice/crio-64c59d151e3bba9d72d84c68425bbb47d72c285e902705aca91708181ed9c2c3 WatchSource:0}: Error finding container 64c59d151e3bba9d72d84c68425bbb47d72c285e902705aca91708181ed9c2c3: Status 404 returned error can't find the container with id 64c59d151e3bba9d72d84c68425bbb47d72c285e902705aca91708181ed9c2c3 Sep 08 15:24:28.906901 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.906872 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:28.986602 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:28.986571 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h"] Sep 08 15:24:28.989447 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:24:28.989400 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3975d31a_b48f_4133_ba5f_72a1d8f11485.slice/crio-0d4a4beb291464e68f0fcc293618c0df921f5386ba5cf2ec72f06bc31ef40add WatchSource:0}: Error finding container 0d4a4beb291464e68f0fcc293618c0df921f5386ba5cf2ec72f06bc31ef40add: Status 404 returned error can't find the container with id 0d4a4beb291464e68f0fcc293618c0df921f5386ba5cf2ec72f06bc31ef40add Sep 08 15:24:29.076922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.076893 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh"] Sep 08 15:24:29.093863 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:24:29.093836 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92bc5b3b_d4bd_4e7b_bfea_f30f45002d6f.slice/crio-a6a03854c627f27514961ac17c594a38e6a7f0d766ee459f4c61e2b6a4ed9f9f WatchSource:0}: Error finding container a6a03854c627f27514961ac17c594a38e6a7f0d766ee459f4c61e2b6a4ed9f9f: Status 404 returned error can't find the container with id a6a03854c627f27514961ac17c594a38e6a7f0d766ee459f4c61e2b6a4ed9f9f Sep 08 15:24:29.196918 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.196887 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb"] Sep 08 15:24:29.251246 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:24:29.251213 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2ce044b_c3c2_4676_aa27_9a013a7d86b4.slice/crio-f5289c0276ee5d3c8550938a87b5473ebb850dd578a14bae48b19ea807032d5a WatchSource:0}: Error finding container f5289c0276ee5d3c8550938a87b5473ebb850dd578a14bae48b19ea807032d5a: Status 404 returned error can't find the container with id f5289c0276ee5d3c8550938a87b5473ebb850dd578a14bae48b19ea807032d5a Sep 08 15:24:29.582744 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.582711 2578 generic.go:356] "Generic (PLEG): container finished" podID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerID="e452ac7d9c53fdf1a6812cae09b2fb807308f7c1f605a79594e58568d8e5deee" exitCode=0 Sep 08 15:24:29.583245 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.582750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" event={"ID":"d04eed1a-f8b1-45e2-9701-946e5b59f25f","Type":"ContainerDied","Data":"e452ac7d9c53fdf1a6812cae09b2fb807308f7c1f605a79594e58568d8e5deee"} Sep 08 15:24:29.583245 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.582811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" event={"ID":"d04eed1a-f8b1-45e2-9701-946e5b59f25f","Type":"ContainerStarted","Data":"64c59d151e3bba9d72d84c68425bbb47d72c285e902705aca91708181ed9c2c3"} Sep 08 15:24:29.584528 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.584504 2578 generic.go:356] "Generic (PLEG): container finished" podID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerID="f5e4d61ccd82d34307bdd50daeb76e049cb2a183cb6fee3adfb7c99a3cd65769" exitCode=0 Sep 08 15:24:29.584702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.584683 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" event={"ID":"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f","Type":"ContainerDied","Data":"f5e4d61ccd82d34307bdd50daeb76e049cb2a183cb6fee3adfb7c99a3cd65769"} Sep 08 15:24:29.584773 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.584732 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" event={"ID":"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f","Type":"ContainerStarted","Data":"a6a03854c627f27514961ac17c594a38e6a7f0d766ee459f4c61e2b6a4ed9f9f"} Sep 08 15:24:29.586915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.586633 2578 generic.go:356] "Generic (PLEG): container finished" podID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerID="80d9152d4f3c170d6636dccf965a6546fec9ebe424407af6187b4c8f9f657310" exitCode=0 Sep 08 15:24:29.586915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.586717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" event={"ID":"3975d31a-b48f-4133-ba5f-72a1d8f11485","Type":"ContainerDied","Data":"80d9152d4f3c170d6636dccf965a6546fec9ebe424407af6187b4c8f9f657310"} Sep 08 15:24:29.586915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.586763 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" event={"ID":"3975d31a-b48f-4133-ba5f-72a1d8f11485","Type":"ContainerStarted","Data":"0d4a4beb291464e68f0fcc293618c0df921f5386ba5cf2ec72f06bc31ef40add"} Sep 08 15:24:29.588568 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.588531 2578 generic.go:356] "Generic (PLEG): container finished" podID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerID="74185eacc5667dd79c76a7dc5ec60eff32888e3b0f7f6834f69a489893362169" exitCode=0 Sep 08 15:24:29.588637 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.588578 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" event={"ID":"d2ce044b-c3c2-4676-aa27-9a013a7d86b4","Type":"ContainerDied","Data":"74185eacc5667dd79c76a7dc5ec60eff32888e3b0f7f6834f69a489893362169"} Sep 08 15:24:29.588637 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:29.588608 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" event={"ID":"d2ce044b-c3c2-4676-aa27-9a013a7d86b4","Type":"ContainerStarted","Data":"f5289c0276ee5d3c8550938a87b5473ebb850dd578a14bae48b19ea807032d5a"} Sep 08 15:24:30.597997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:30.597960 2578 generic.go:356] "Generic (PLEG): container finished" podID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerID="3004099ead701d063c39b3bcea58b3ae96756fe9c4ad85767dbc3e9f815a038d" exitCode=0 Sep 08 15:24:30.598354 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:30.598023 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" event={"ID":"3975d31a-b48f-4133-ba5f-72a1d8f11485","Type":"ContainerDied","Data":"3004099ead701d063c39b3bcea58b3ae96756fe9c4ad85767dbc3e9f815a038d"} Sep 08 15:24:31.606473 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:31.606434 2578 generic.go:356] "Generic (PLEG): container finished" podID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerID="b6fa1434a368634197b23423f7c4ed992cb4d8a5ec1d9eaa9bb692224353d693" exitCode=0 Sep 08 15:24:31.606901 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:31.606520 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" event={"ID":"d2ce044b-c3c2-4676-aa27-9a013a7d86b4","Type":"ContainerDied","Data":"b6fa1434a368634197b23423f7c4ed992cb4d8a5ec1d9eaa9bb692224353d693"} Sep 08 15:24:31.608870 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:31.608841 2578 generic.go:356] "Generic (PLEG): container finished" podID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerID="1b37b6f1ba01a3011876baa0784259f3d9ce43122a643418a6be233a831a829b" exitCode=0 Sep 08 15:24:31.608998 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:31.608972 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" event={"ID":"d04eed1a-f8b1-45e2-9701-946e5b59f25f","Type":"ContainerDied","Data":"1b37b6f1ba01a3011876baa0784259f3d9ce43122a643418a6be233a831a829b"} Sep 08 15:24:31.611270 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:31.611241 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" event={"ID":"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f","Type":"ContainerStarted","Data":"301189cbd7c1023e72622fcf51051d3d8bb3730e45e32e407607579fe591613c"} Sep 08 15:24:31.614008 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:31.613980 2578 generic.go:356] "Generic (PLEG): container finished" podID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerID="15421952c5a0ac7b4ad336fa4c929dd6fc338d296b4e21fa514a7fe0adb6e1d7" exitCode=0 Sep 08 15:24:31.614141 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:31.614026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" event={"ID":"3975d31a-b48f-4133-ba5f-72a1d8f11485","Type":"ContainerDied","Data":"15421952c5a0ac7b4ad336fa4c929dd6fc338d296b4e21fa514a7fe0adb6e1d7"} Sep 08 15:24:32.627421 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.627380 2578 generic.go:356] "Generic (PLEG): container finished" podID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerID="5bf512d12ffe675738377abba3cdb30462560627c359c401b39ef7c22c13ec8f" exitCode=0 Sep 08 15:24:32.627780 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.627465 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" event={"ID":"d04eed1a-f8b1-45e2-9701-946e5b59f25f","Type":"ContainerDied","Data":"5bf512d12ffe675738377abba3cdb30462560627c359c401b39ef7c22c13ec8f"} Sep 08 15:24:32.629292 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.629269 2578 generic.go:356] "Generic (PLEG): container finished" podID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerID="301189cbd7c1023e72622fcf51051d3d8bb3730e45e32e407607579fe591613c" exitCode=0 Sep 08 15:24:32.629416 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.629351 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" event={"ID":"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f","Type":"ContainerDied","Data":"301189cbd7c1023e72622fcf51051d3d8bb3730e45e32e407607579fe591613c"} Sep 08 15:24:32.631687 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.631664 2578 generic.go:356] "Generic (PLEG): container finished" podID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerID="cb3372011ba64a2c0da2ef36768f2032924bee858593a3d205ab0bc6e8e7380c" exitCode=0 Sep 08 15:24:32.631794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.631734 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" event={"ID":"d2ce044b-c3c2-4676-aa27-9a013a7d86b4","Type":"ContainerDied","Data":"cb3372011ba64a2c0da2ef36768f2032924bee858593a3d205ab0bc6e8e7380c"} Sep 08 15:24:32.769837 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.769770 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:32.847263 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.847220 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-bundle\") pod \"3975d31a-b48f-4133-ba5f-72a1d8f11485\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " Sep 08 15:24:32.847444 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.847317 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mmj8\" (UniqueName: \"kubernetes.io/projected/3975d31a-b48f-4133-ba5f-72a1d8f11485-kube-api-access-8mmj8\") pod \"3975d31a-b48f-4133-ba5f-72a1d8f11485\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " Sep 08 15:24:32.847444 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.847355 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-util\") pod \"3975d31a-b48f-4133-ba5f-72a1d8f11485\" (UID: \"3975d31a-b48f-4133-ba5f-72a1d8f11485\") " Sep 08 15:24:32.847947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.847903 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-bundle" (OuterVolumeSpecName: "bundle") pod "3975d31a-b48f-4133-ba5f-72a1d8f11485" (UID: "3975d31a-b48f-4133-ba5f-72a1d8f11485"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:24:32.849550 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.849526 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3975d31a-b48f-4133-ba5f-72a1d8f11485-kube-api-access-8mmj8" (OuterVolumeSpecName: "kube-api-access-8mmj8") pod "3975d31a-b48f-4133-ba5f-72a1d8f11485" (UID: "3975d31a-b48f-4133-ba5f-72a1d8f11485"). InnerVolumeSpecName "kube-api-access-8mmj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:24:32.852755 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.852724 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-util" (OuterVolumeSpecName: "util") pod "3975d31a-b48f-4133-ba5f-72a1d8f11485" (UID: "3975d31a-b48f-4133-ba5f-72a1d8f11485"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:24:32.948382 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.948338 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:32.948382 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.948374 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mmj8\" (UniqueName: \"kubernetes.io/projected/3975d31a-b48f-4133-ba5f-72a1d8f11485-kube-api-access-8mmj8\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:32.948382 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:32.948384 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3975d31a-b48f-4133-ba5f-72a1d8f11485-util\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:33.639356 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.639314 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" Sep 08 15:24:33.639356 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.639347 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053glr5h" event={"ID":"3975d31a-b48f-4133-ba5f-72a1d8f11485","Type":"ContainerDied","Data":"0d4a4beb291464e68f0fcc293618c0df921f5386ba5cf2ec72f06bc31ef40add"} Sep 08 15:24:33.639852 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.639401 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d4a4beb291464e68f0fcc293618c0df921f5386ba5cf2ec72f06bc31ef40add" Sep 08 15:24:33.641775 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.641746 2578 generic.go:356] "Generic (PLEG): container finished" podID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerID="020c7c14d09ccdb8cc7cb74a6e5defff5b3645add15ea557479ca775711196c5" exitCode=0 Sep 08 15:24:33.641903 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.641820 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" event={"ID":"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f","Type":"ContainerDied","Data":"020c7c14d09ccdb8cc7cb74a6e5defff5b3645add15ea557479ca775711196c5"} Sep 08 15:24:33.798364 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.798340 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:33.801748 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.801729 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:33.858593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.858557 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-bundle\") pod \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " Sep 08 15:24:33.858593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.858598 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-util\") pod \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " Sep 08 15:24:33.858870 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.858623 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2rvd\" (UniqueName: \"kubernetes.io/projected/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-kube-api-access-k2rvd\") pod \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\" (UID: \"d2ce044b-c3c2-4676-aa27-9a013a7d86b4\") " Sep 08 15:24:33.859046 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.859024 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-bundle" (OuterVolumeSpecName: "bundle") pod "d2ce044b-c3c2-4676-aa27-9a013a7d86b4" (UID: "d2ce044b-c3c2-4676-aa27-9a013a7d86b4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:24:33.860927 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.860892 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-kube-api-access-k2rvd" (OuterVolumeSpecName: "kube-api-access-k2rvd") pod "d2ce044b-c3c2-4676-aa27-9a013a7d86b4" (UID: "d2ce044b-c3c2-4676-aa27-9a013a7d86b4"). InnerVolumeSpecName "kube-api-access-k2rvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:24:33.863880 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.863852 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-util" (OuterVolumeSpecName: "util") pod "d2ce044b-c3c2-4676-aa27-9a013a7d86b4" (UID: "d2ce044b-c3c2-4676-aa27-9a013a7d86b4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:24:33.960275 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.960234 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g27w2\" (UniqueName: \"kubernetes.io/projected/d04eed1a-f8b1-45e2-9701-946e5b59f25f-kube-api-access-g27w2\") pod \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " Sep 08 15:24:33.960453 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.960303 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-util\") pod \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " Sep 08 15:24:33.960453 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.960340 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-bundle\") pod \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\" (UID: \"d04eed1a-f8b1-45e2-9701-946e5b59f25f\") " Sep 08 15:24:33.960652 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.960634 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:33.960694 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.960658 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-util\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:33.960694 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.960671 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2rvd\" (UniqueName: \"kubernetes.io/projected/d2ce044b-c3c2-4676-aa27-9a013a7d86b4-kube-api-access-k2rvd\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:33.960953 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.960929 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-bundle" (OuterVolumeSpecName: "bundle") pod "d04eed1a-f8b1-45e2-9701-946e5b59f25f" (UID: "d04eed1a-f8b1-45e2-9701-946e5b59f25f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:24:33.962707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.962680 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d04eed1a-f8b1-45e2-9701-946e5b59f25f-kube-api-access-g27w2" (OuterVolumeSpecName: "kube-api-access-g27w2") pod "d04eed1a-f8b1-45e2-9701-946e5b59f25f" (UID: "d04eed1a-f8b1-45e2-9701-946e5b59f25f"). InnerVolumeSpecName "kube-api-access-g27w2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:24:33.965750 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:33.965697 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-util" (OuterVolumeSpecName: "util") pod "d04eed1a-f8b1-45e2-9701-946e5b59f25f" (UID: "d04eed1a-f8b1-45e2-9701-946e5b59f25f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:24:34.061711 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.061613 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g27w2\" (UniqueName: \"kubernetes.io/projected/d04eed1a-f8b1-45e2-9701-946e5b59f25f-kube-api-access-g27w2\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:34.061711 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.061645 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-util\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:34.061711 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.061656 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d04eed1a-f8b1-45e2-9701-946e5b59f25f-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:34.649328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.649297 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" Sep 08 15:24:34.649328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.649307 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0gd2vb" event={"ID":"d2ce044b-c3c2-4676-aa27-9a013a7d86b4","Type":"ContainerDied","Data":"f5289c0276ee5d3c8550938a87b5473ebb850dd578a14bae48b19ea807032d5a"} Sep 08 15:24:34.649806 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.649356 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5289c0276ee5d3c8550938a87b5473ebb850dd578a14bae48b19ea807032d5a" Sep 08 15:24:34.651430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.651409 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" Sep 08 15:24:34.651536 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.651435 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnwkhg" event={"ID":"d04eed1a-f8b1-45e2-9701-946e5b59f25f","Type":"ContainerDied","Data":"64c59d151e3bba9d72d84c68425bbb47d72c285e902705aca91708181ed9c2c3"} Sep 08 15:24:34.651536 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.651479 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64c59d151e3bba9d72d84c68425bbb47d72c285e902705aca91708181ed9c2c3" Sep 08 15:24:34.784290 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.784228 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:34.869459 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.869417 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-util\") pod \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " Sep 08 15:24:34.869640 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.869493 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59vhv\" (UniqueName: \"kubernetes.io/projected/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-kube-api-access-59vhv\") pod \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " Sep 08 15:24:34.869640 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.869592 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-bundle\") pod \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\" (UID: \"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f\") " Sep 08 15:24:34.870160 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.870095 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-bundle" (OuterVolumeSpecName: "bundle") pod "92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" (UID: "92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:24:34.871800 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.871778 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-kube-api-access-59vhv" (OuterVolumeSpecName: "kube-api-access-59vhv") pod "92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" (UID: "92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f"). InnerVolumeSpecName "kube-api-access-59vhv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:24:34.875379 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.875352 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-util" (OuterVolumeSpecName: "util") pod "92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" (UID: "92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:24:34.971311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.971273 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-util\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:34.971311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.971308 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59vhv\" (UniqueName: \"kubernetes.io/projected/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-kube-api-access-59vhv\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:34.971311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:34.971319 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:35.659219 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:35.659191 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" Sep 08 15:24:35.659219 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:35.659206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbdzzzh" event={"ID":"92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f","Type":"ContainerDied","Data":"a6a03854c627f27514961ac17c594a38e6a7f0d766ee459f4c61e2b6a4ed9f9f"} Sep 08 15:24:35.659625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:35.659250 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6a03854c627f27514961ac17c594a38e6a7f0d766ee459f4c61e2b6a4ed9f9f" Sep 08 15:24:41.598521 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.598457 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-554d477fb-ms2w6" podUID="07b1c0e2-eab9-4a3d-803e-6f6243cbb598" containerName="console" containerID="cri-o://d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346" gracePeriod=15 Sep 08 15:24:41.852518 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.852452 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:24:41.935101 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935061 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-oauth-config\") pod \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " Sep 08 15:24:41.935311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935141 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-config\") pod \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " Sep 08 15:24:41.935311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935175 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxbpk\" (UniqueName: \"kubernetes.io/projected/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-kube-api-access-jxbpk\") pod \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " Sep 08 15:24:41.935311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935212 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-service-ca\") pod \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " Sep 08 15:24:41.935311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935244 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-serving-cert\") pod \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " Sep 08 15:24:41.935520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935348 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-oauth-serving-cert\") pod \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " Sep 08 15:24:41.935520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935384 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-trusted-ca-bundle\") pod \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\" (UID: \"07b1c0e2-eab9-4a3d-803e-6f6243cbb598\") " Sep 08 15:24:41.935671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935638 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-config" (OuterVolumeSpecName: "console-config") pod "07b1c0e2-eab9-4a3d-803e-6f6243cbb598" (UID: "07b1c0e2-eab9-4a3d-803e-6f6243cbb598"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:24:41.935671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935643 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-service-ca" (OuterVolumeSpecName: "service-ca") pod "07b1c0e2-eab9-4a3d-803e-6f6243cbb598" (UID: "07b1c0e2-eab9-4a3d-803e-6f6243cbb598"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:24:41.935772 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935717 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "07b1c0e2-eab9-4a3d-803e-6f6243cbb598" (UID: "07b1c0e2-eab9-4a3d-803e-6f6243cbb598"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:24:41.935772 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.935741 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-config\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:41.936257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.936231 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "07b1c0e2-eab9-4a3d-803e-6f6243cbb598" (UID: "07b1c0e2-eab9-4a3d-803e-6f6243cbb598"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:24:41.937581 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.937551 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "07b1c0e2-eab9-4a3d-803e-6f6243cbb598" (UID: "07b1c0e2-eab9-4a3d-803e-6f6243cbb598"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:24:41.937581 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.937565 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "07b1c0e2-eab9-4a3d-803e-6f6243cbb598" (UID: "07b1c0e2-eab9-4a3d-803e-6f6243cbb598"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 15:24:41.937713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:41.937598 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-kube-api-access-jxbpk" (OuterVolumeSpecName: "kube-api-access-jxbpk") pod "07b1c0e2-eab9-4a3d-803e-6f6243cbb598" (UID: "07b1c0e2-eab9-4a3d-803e-6f6243cbb598"). InnerVolumeSpecName "kube-api-access-jxbpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:24:42.036610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.036561 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:42.036610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.036613 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-serving-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:42.036823 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.036629 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-oauth-serving-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:42.036823 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.036642 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-trusted-ca-bundle\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:42.036823 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.036654 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-console-oauth-config\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:42.036823 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.036665 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxbpk\" (UniqueName: \"kubernetes.io/projected/07b1c0e2-eab9-4a3d-803e-6f6243cbb598-kube-api-access-jxbpk\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:24:42.697708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.697674 2578 generic.go:356] "Generic (PLEG): container finished" podID="07b1c0e2-eab9-4a3d-803e-6f6243cbb598" containerID="d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346" exitCode=0 Sep 08 15:24:42.698106 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.697737 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554d477fb-ms2w6" Sep 08 15:24:42.698106 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.697767 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554d477fb-ms2w6" event={"ID":"07b1c0e2-eab9-4a3d-803e-6f6243cbb598","Type":"ContainerDied","Data":"d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346"} Sep 08 15:24:42.698106 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.697821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554d477fb-ms2w6" event={"ID":"07b1c0e2-eab9-4a3d-803e-6f6243cbb598","Type":"ContainerDied","Data":"8235b748e4fd765ac19ca52caf12e1e80049c952d7d2c5525861210b21e21fea"} Sep 08 15:24:42.698106 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.697846 2578 scope.go:117] "RemoveContainer" containerID="d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346" Sep 08 15:24:42.707702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.707680 2578 scope.go:117] "RemoveContainer" containerID="d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346" Sep 08 15:24:42.708013 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:24:42.707994 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346\": container with ID starting with d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346 not found: ID does not exist" containerID="d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346" Sep 08 15:24:42.708067 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.708022 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346"} err="failed to get container status \"d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346\": rpc error: code = NotFound desc = could not find container \"d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346\": container with ID starting with d5f45e5825e008ffb7f035d6112831a98d0e4ead6119f8d72954c1976f3f6346 not found: ID does not exist" Sep 08 15:24:42.740434 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.740398 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554d477fb-ms2w6"] Sep 08 15:24:42.752664 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:42.752630 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-554d477fb-ms2w6"] Sep 08 15:24:43.502488 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:43.502453 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b1c0e2-eab9-4a3d-803e-6f6243cbb598" path="/var/lib/kubelet/pods/07b1c0e2-eab9-4a3d-803e-6f6243cbb598/volumes" Sep 08 15:24:44.414820 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.414784 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-4stlk"] Sep 08 15:24:44.415210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415175 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerName="extract" Sep 08 15:24:44.415210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415189 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerName="extract" Sep 08 15:24:44.415210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415198 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerName="extract" Sep 08 15:24:44.415210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415204 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerName="extract" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415215 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b1c0e2-eab9-4a3d-803e-6f6243cbb598" containerName="console" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415221 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b1c0e2-eab9-4a3d-803e-6f6243cbb598" containerName="console" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415226 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerName="pull" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415231 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerName="pull" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415244 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerName="extract" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415249 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerName="extract" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415255 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerName="util" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415261 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerName="util" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415266 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerName="util" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415271 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerName="util" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415276 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerName="util" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415281 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerName="util" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415286 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerName="extract" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415290 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerName="extract" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415300 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerName="util" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415305 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerName="util" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415311 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerName="pull" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415315 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerName="pull" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415321 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerName="pull" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415326 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerName="pull" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415332 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerName="pull" Sep 08 15:24:44.415340 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415337 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerName="pull" Sep 08 15:24:44.415946 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415389 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b1c0e2-eab9-4a3d-803e-6f6243cbb598" containerName="console" Sep 08 15:24:44.415946 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415396 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3975d31a-b48f-4133-ba5f-72a1d8f11485" containerName="extract" Sep 08 15:24:44.415946 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415403 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="92bc5b3b-d4bd-4e7b-bfea-f30f45002d6f" containerName="extract" Sep 08 15:24:44.415946 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415411 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d04eed1a-f8b1-45e2-9701-946e5b59f25f" containerName="extract" Sep 08 15:24:44.415946 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.415416 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2ce044b-c3c2-4676-aa27-9a013a7d86b4" containerName="extract" Sep 08 15:24:44.419859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.419840 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" Sep 08 15:24:44.422590 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.422566 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 08 15:24:44.422727 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.422577 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hlxxx\"" Sep 08 15:24:44.422727 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.422655 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 08 15:24:44.422727 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.422615 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 08 15:24:44.428483 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.428461 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-4stlk"] Sep 08 15:24:44.560163 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.560128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz659\" (UniqueName: \"kubernetes.io/projected/02326afe-568c-4990-9780-c3e68e1f47d1-kube-api-access-mz659\") pod \"dns-operator-controller-manager-5985564776-4stlk\" (UID: \"02326afe-568c-4990-9780-c3e68e1f47d1\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" Sep 08 15:24:44.661568 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.661530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz659\" (UniqueName: \"kubernetes.io/projected/02326afe-568c-4990-9780-c3e68e1f47d1-kube-api-access-mz659\") pod \"dns-operator-controller-manager-5985564776-4stlk\" (UID: \"02326afe-568c-4990-9780-c3e68e1f47d1\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" Sep 08 15:24:44.680398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.680313 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz659\" (UniqueName: \"kubernetes.io/projected/02326afe-568c-4990-9780-c3e68e1f47d1-kube-api-access-mz659\") pod \"dns-operator-controller-manager-5985564776-4stlk\" (UID: \"02326afe-568c-4990-9780-c3e68e1f47d1\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" Sep 08 15:24:44.731284 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:44.731249 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" Sep 08 15:24:45.039263 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:45.039230 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-4stlk"] Sep 08 15:24:45.040258 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:24:45.040230 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02326afe_568c_4990_9780_c3e68e1f47d1.slice/crio-305a555873da0311376d2d5f1fd333db9196d3769806aec738ec99915c64d0ca WatchSource:0}: Error finding container 305a555873da0311376d2d5f1fd333db9196d3769806aec738ec99915c64d0ca: Status 404 returned error can't find the container with id 305a555873da0311376d2d5f1fd333db9196d3769806aec738ec99915c64d0ca Sep 08 15:24:45.714997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:45.714961 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" event={"ID":"02326afe-568c-4990-9780-c3e68e1f47d1","Type":"ContainerStarted","Data":"305a555873da0311376d2d5f1fd333db9196d3769806aec738ec99915c64d0ca"} Sep 08 15:24:47.331917 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:47.331876 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-g7vj7"] Sep 08 15:24:47.341371 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:47.341337 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" Sep 08 15:24:47.345462 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:47.345405 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-44ds8\"" Sep 08 15:24:47.356897 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:47.356873 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-g7vj7"] Sep 08 15:24:47.488665 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:47.488614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9mtl\" (UniqueName: \"kubernetes.io/projected/264af2dd-1e4d-46dd-9467-b5d1d232903b-kube-api-access-f9mtl\") pod \"authorino-operator-68d88948d8-g7vj7\" (UID: \"264af2dd-1e4d-46dd-9467-b5d1d232903b\") " pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" Sep 08 15:24:47.589563 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:47.589483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9mtl\" (UniqueName: \"kubernetes.io/projected/264af2dd-1e4d-46dd-9467-b5d1d232903b-kube-api-access-f9mtl\") pod \"authorino-operator-68d88948d8-g7vj7\" (UID: \"264af2dd-1e4d-46dd-9467-b5d1d232903b\") " pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" Sep 08 15:24:47.599720 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:47.599689 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9mtl\" (UniqueName: \"kubernetes.io/projected/264af2dd-1e4d-46dd-9467-b5d1d232903b-kube-api-access-f9mtl\") pod \"authorino-operator-68d88948d8-g7vj7\" (UID: \"264af2dd-1e4d-46dd-9467-b5d1d232903b\") " pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" Sep 08 15:24:47.656070 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:47.656032 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" Sep 08 15:24:48.458388 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:48.458351 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-g7vj7"] Sep 08 15:24:48.461366 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:24:48.461338 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264af2dd_1e4d_46dd_9467_b5d1d232903b.slice/crio-b3cd5f45f16d51cce187da8108d87ce52b4da3ff1062cbac424ed718ac492b56 WatchSource:0}: Error finding container b3cd5f45f16d51cce187da8108d87ce52b4da3ff1062cbac424ed718ac492b56: Status 404 returned error can't find the container with id b3cd5f45f16d51cce187da8108d87ce52b4da3ff1062cbac424ed718ac492b56 Sep 08 15:24:48.738159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:48.738047 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" event={"ID":"02326afe-568c-4990-9780-c3e68e1f47d1","Type":"ContainerStarted","Data":"ddff6ecb7ed1dce83dbc3f117f1acef4f8b438db95ba77918bc0df50480cd0c9"} Sep 08 15:24:48.738333 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:48.738178 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" Sep 08 15:24:48.739500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:48.739480 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" event={"ID":"264af2dd-1e4d-46dd-9467-b5d1d232903b","Type":"ContainerStarted","Data":"b3cd5f45f16d51cce187da8108d87ce52b4da3ff1062cbac424ed718ac492b56"} Sep 08 15:24:48.764816 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:48.764768 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" podStartSLOduration=1.558576607 podStartE2EDuration="4.764753106s" totalImagesPullingTime="3.206176499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:24:48.763916602 +0000 UTC m=+427.787541059" watchObservedRunningTime="2026-09-08 15:24:48.764753106 +0000 UTC m=+427.788377562" Sep 08 15:24:50.754687 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:50.754650 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" event={"ID":"264af2dd-1e4d-46dd-9467-b5d1d232903b","Type":"ContainerStarted","Data":"6e4137f816d34ee62411db9bd1e83b24fb65d7ee5e8057ccf20d4f05084c3617"} Sep 08 15:24:50.755086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:50.754776 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" Sep 08 15:24:50.775515 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:50.775465 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" podStartSLOduration=2.064257704 podStartE2EDuration="3.77544935s" totalImagesPullingTime="1.711191646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:24:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:24:50.773232427 +0000 UTC m=+429.796856884" watchObservedRunningTime="2026-09-08 15:24:50.77544935 +0000 UTC m=+429.799073805" Sep 08 15:24:59.748828 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:24:59.748789 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4stlk" Sep 08 15:25:01.762779 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:01.762746 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-g7vj7" Sep 08 15:25:01.981487 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:01.981452 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b"] Sep 08 15:25:01.985750 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:01.985723 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" Sep 08 15:25:01.989065 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:01.989044 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5bk8c\"" Sep 08 15:25:01.997418 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:01.997384 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b"] Sep 08 15:25:02.015025 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:02.014940 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bvfw\" (UniqueName: \"kubernetes.io/projected/71ecf2da-5125-400c-8b95-8de4eee2055d-kube-api-access-4bvfw\") pod \"limitador-operator-controller-manager-6c549b6d85-wc64b\" (UID: \"71ecf2da-5125-400c-8b95-8de4eee2055d\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" Sep 08 15:25:02.115655 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:02.115606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bvfw\" (UniqueName: \"kubernetes.io/projected/71ecf2da-5125-400c-8b95-8de4eee2055d-kube-api-access-4bvfw\") pod \"limitador-operator-controller-manager-6c549b6d85-wc64b\" (UID: \"71ecf2da-5125-400c-8b95-8de4eee2055d\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" Sep 08 15:25:02.130914 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:02.130879 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bvfw\" (UniqueName: \"kubernetes.io/projected/71ecf2da-5125-400c-8b95-8de4eee2055d-kube-api-access-4bvfw\") pod \"limitador-operator-controller-manager-6c549b6d85-wc64b\" (UID: \"71ecf2da-5125-400c-8b95-8de4eee2055d\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" Sep 08 15:25:02.297180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:02.297045 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" Sep 08 15:25:02.581396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:02.581357 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b"] Sep 08 15:25:02.582876 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:25:02.582847 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ecf2da_5125_400c_8b95_8de4eee2055d.slice/crio-948e1adea34082baebda938c5e096a81bb98dc61f3922bab81d06eb2e2401efc WatchSource:0}: Error finding container 948e1adea34082baebda938c5e096a81bb98dc61f3922bab81d06eb2e2401efc: Status 404 returned error can't find the container with id 948e1adea34082baebda938c5e096a81bb98dc61f3922bab81d06eb2e2401efc Sep 08 15:25:02.827616 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:02.827577 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" event={"ID":"71ecf2da-5125-400c-8b95-8de4eee2055d","Type":"ContainerStarted","Data":"948e1adea34082baebda938c5e096a81bb98dc61f3922bab81d06eb2e2401efc"} Sep 08 15:25:04.841439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:04.841393 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" event={"ID":"71ecf2da-5125-400c-8b95-8de4eee2055d","Type":"ContainerStarted","Data":"bb593dc38b65f5e6304a1c36627b782e7b200c18b1a4df3b5b8dd262e4a907ad"} Sep 08 15:25:04.841867 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:04.841475 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" Sep 08 15:25:04.862308 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:04.862248 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" podStartSLOduration=2.173331071 podStartE2EDuration="3.862230461s" totalImagesPullingTime="1.68889939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:25:04.860669161 +0000 UTC m=+443.884293619" watchObservedRunningTime="2026-09-08 15:25:04.862230461 +0000 UTC m=+443.885854917" Sep 08 15:25:15.849878 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:15.849845 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wc64b" Sep 08 15:25:44.721411 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:44.721370 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8pvb9"] Sep 08 15:25:44.727682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:44.727659 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" Sep 08 15:25:44.730633 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:44.730603 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rvx67\"" Sep 08 15:25:44.732544 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:44.732520 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8pvb9"] Sep 08 15:25:44.790174 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:44.790141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlwkl\" (UniqueName: \"kubernetes.io/projected/a13704ff-0a4d-4dcf-81da-2cbfcde5f249-kube-api-access-tlwkl\") pod \"authorino-c548b5d7d-8pvb9\" (UID: \"a13704ff-0a4d-4dcf-81da-2cbfcde5f249\") " pod="kuadrant-system/authorino-c548b5d7d-8pvb9" Sep 08 15:25:44.891266 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:44.891226 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwkl\" (UniqueName: \"kubernetes.io/projected/a13704ff-0a4d-4dcf-81da-2cbfcde5f249-kube-api-access-tlwkl\") pod \"authorino-c548b5d7d-8pvb9\" (UID: \"a13704ff-0a4d-4dcf-81da-2cbfcde5f249\") " pod="kuadrant-system/authorino-c548b5d7d-8pvb9" Sep 08 15:25:44.900822 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:44.900791 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlwkl\" (UniqueName: \"kubernetes.io/projected/a13704ff-0a4d-4dcf-81da-2cbfcde5f249-kube-api-access-tlwkl\") pod \"authorino-c548b5d7d-8pvb9\" (UID: \"a13704ff-0a4d-4dcf-81da-2cbfcde5f249\") " pod="kuadrant-system/authorino-c548b5d7d-8pvb9" Sep 08 15:25:45.038957 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:45.038864 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" Sep 08 15:25:45.316300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:45.316274 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8pvb9"] Sep 08 15:25:45.318822 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:25:45.318794 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda13704ff_0a4d_4dcf_81da_2cbfcde5f249.slice/crio-a21dd08b3bf9c6e3127e3563dfcbe391306987a15fea8a0753da5ccc9f127e90 WatchSource:0}: Error finding container a21dd08b3bf9c6e3127e3563dfcbe391306987a15fea8a0753da5ccc9f127e90: Status 404 returned error can't find the container with id a21dd08b3bf9c6e3127e3563dfcbe391306987a15fea8a0753da5ccc9f127e90 Sep 08 15:25:46.077011 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:46.076975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" event={"ID":"a13704ff-0a4d-4dcf-81da-2cbfcde5f249","Type":"ContainerStarted","Data":"a21dd08b3bf9c6e3127e3563dfcbe391306987a15fea8a0753da5ccc9f127e90"} Sep 08 15:25:49.101237 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:49.101195 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" event={"ID":"a13704ff-0a4d-4dcf-81da-2cbfcde5f249","Type":"ContainerStarted","Data":"5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979"} Sep 08 15:25:49.119959 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:25:49.119899 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" podStartSLOduration=2.3509340180000002 podStartE2EDuration="5.119885235s" totalImagesPullingTime="2.768951217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:25:49.118399649 +0000 UTC m=+488.142024108" watchObservedRunningTime="2026-09-08 15:25:49.119885235 +0000 UTC m=+488.143509691" Sep 08 15:26:18.652692 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:18.652647 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8pvb9"] Sep 08 15:26:18.653243 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:18.652847 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" podUID="a13704ff-0a4d-4dcf-81da-2cbfcde5f249" containerName="authorino" containerID="cri-o://5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979" gracePeriod=30 Sep 08 15:26:18.914642 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:18.914555 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" Sep 08 15:26:19.001722 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.001686 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlwkl\" (UniqueName: \"kubernetes.io/projected/a13704ff-0a4d-4dcf-81da-2cbfcde5f249-kube-api-access-tlwkl\") pod \"a13704ff-0a4d-4dcf-81da-2cbfcde5f249\" (UID: \"a13704ff-0a4d-4dcf-81da-2cbfcde5f249\") " Sep 08 15:26:19.004058 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.004025 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a13704ff-0a4d-4dcf-81da-2cbfcde5f249-kube-api-access-tlwkl" (OuterVolumeSpecName: "kube-api-access-tlwkl") pod "a13704ff-0a4d-4dcf-81da-2cbfcde5f249" (UID: "a13704ff-0a4d-4dcf-81da-2cbfcde5f249"). InnerVolumeSpecName "kube-api-access-tlwkl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:26:19.103257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.103216 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlwkl\" (UniqueName: \"kubernetes.io/projected/a13704ff-0a4d-4dcf-81da-2cbfcde5f249-kube-api-access-tlwkl\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:26:19.288747 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.288610 2578 generic.go:356] "Generic (PLEG): container finished" podID="a13704ff-0a4d-4dcf-81da-2cbfcde5f249" containerID="5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979" exitCode=0 Sep 08 15:26:19.288747 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.288675 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" Sep 08 15:26:19.288747 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.288727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" event={"ID":"a13704ff-0a4d-4dcf-81da-2cbfcde5f249","Type":"ContainerDied","Data":"5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979"} Sep 08 15:26:19.289061 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.288790 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8pvb9" event={"ID":"a13704ff-0a4d-4dcf-81da-2cbfcde5f249","Type":"ContainerDied","Data":"a21dd08b3bf9c6e3127e3563dfcbe391306987a15fea8a0753da5ccc9f127e90"} Sep 08 15:26:19.289061 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.288815 2578 scope.go:117] "RemoveContainer" containerID="5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979" Sep 08 15:26:19.301811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.301784 2578 scope.go:117] "RemoveContainer" containerID="5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979" Sep 08 15:26:19.302252 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:26:19.302226 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979\": container with ID starting with 5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979 not found: ID does not exist" containerID="5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979" Sep 08 15:26:19.302356 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.302263 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979"} err="failed to get container status \"5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979\": rpc error: code = NotFound desc = could not find container \"5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979\": container with ID starting with 5574040417c283012d813b626230803a8596e7d9e068613a323b57db2ca67979 not found: ID does not exist" Sep 08 15:26:19.317756 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.317724 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8pvb9"] Sep 08 15:26:19.325130 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.325088 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8pvb9"] Sep 08 15:26:19.502033 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:26:19.501999 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a13704ff-0a4d-4dcf-81da-2cbfcde5f249" path="/var/lib/kubelet/pods/a13704ff-0a4d-4dcf-81da-2cbfcde5f249/volumes" Sep 08 15:27:17.411819 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.411778 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75"] Sep 08 15:27:17.412450 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.412432 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a13704ff-0a4d-4dcf-81da-2cbfcde5f249" containerName="authorino" Sep 08 15:27:17.412499 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.412453 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a13704ff-0a4d-4dcf-81da-2cbfcde5f249" containerName="authorino" Sep 08 15:27:17.412559 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.412546 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a13704ff-0a4d-4dcf-81da-2cbfcde5f249" containerName="authorino" Sep 08 15:27:17.416186 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.416160 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.420806 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.420766 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 08 15:27:17.420806 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.420768 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tmrxc\"" Sep 08 15:27:17.421025 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.420769 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 08 15:27:17.421025 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.420776 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 08 15:27:17.426653 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.426625 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75"] Sep 08 15:27:17.533060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.533016 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.533259 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.533082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.533259 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.533171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.533259 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.533221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.533362 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.533284 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7brts\" (UniqueName: \"kubernetes.io/projected/42568245-ed3d-432d-824d-452457c0cc34-kube-api-access-7brts\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.533362 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.533331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42568245-ed3d-432d-824d-452457c0cc34-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.634649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.634615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7brts\" (UniqueName: \"kubernetes.io/projected/42568245-ed3d-432d-824d-452457c0cc34-kube-api-access-7brts\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.634649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.634650 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42568245-ed3d-432d-824d-452457c0cc34-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.634891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.634687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.634891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.634714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.634891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.634752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.634891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.634782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.635171 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.635147 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.635236 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.635169 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.635276 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.635226 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.637356 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.637330 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42568245-ed3d-432d-824d-452457c0cc34-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.637560 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.637540 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42568245-ed3d-432d-824d-452457c0cc34-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.644518 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.644493 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7brts\" (UniqueName: \"kubernetes.io/projected/42568245-ed3d-432d-824d-452457c0cc34-kube-api-access-7brts\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75\" (UID: \"42568245-ed3d-432d-824d-452457c0cc34\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:17.729429 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:17.729386 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:18.005169 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:18.005142 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75"] Sep 08 15:27:18.007197 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:27:18.007144 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42568245_ed3d_432d_824d_452457c0cc34.slice/crio-0d9a172877bdfc1d68458d286963d8cc1f013b9a2e484f82b7589d22f3baa561 WatchSource:0}: Error finding container 0d9a172877bdfc1d68458d286963d8cc1f013b9a2e484f82b7589d22f3baa561: Status 404 returned error can't find the container with id 0d9a172877bdfc1d68458d286963d8cc1f013b9a2e484f82b7589d22f3baa561 Sep 08 15:27:18.647697 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:18.647656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" event={"ID":"42568245-ed3d-432d-824d-452457c0cc34","Type":"ContainerStarted","Data":"0d9a172877bdfc1d68458d286963d8cc1f013b9a2e484f82b7589d22f3baa561"} Sep 08 15:27:19.654728 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:19.654687 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" event={"ID":"42568245-ed3d-432d-824d-452457c0cc34","Type":"ContainerStarted","Data":"0272f612e982c930c307095b71f1078aae83c2113c7fa2d52f478e472f64bf67"} Sep 08 15:27:19.677034 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:19.676981 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" podStartSLOduration=1.493723901 podStartE2EDuration="2.676965148s" totalImagesPullingTime="1.183241247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:19.674594908 +0000 UTC m=+578.698219365" watchObservedRunningTime="2026-09-08 15:27:19.676965148 +0000 UTC m=+578.700589604" Sep 08 15:27:24.703654 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.703619 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz"] Sep 08 15:27:24.707306 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.707290 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.710138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.710099 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 08 15:27:24.716393 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.716370 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz"] Sep 08 15:27:24.808245 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.808208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pdp6\" (UniqueName: \"kubernetes.io/projected/efb86b57-5e55-4a1d-a9be-5f674fd80902-kube-api-access-9pdp6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.808439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.808263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.808439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.808298 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.808439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.808349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb86b57-5e55-4a1d-a9be-5f674fd80902-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.808439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.808416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.808626 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.808486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.909285 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.909233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.909285 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.909285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.909543 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.909313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb86b57-5e55-4a1d-a9be-5f674fd80902-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.909543 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.909386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.909543 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.909410 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.909543 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.909491 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pdp6\" (UniqueName: \"kubernetes.io/projected/efb86b57-5e55-4a1d-a9be-5f674fd80902-kube-api-access-9pdp6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.909736 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.909706 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.909793 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.909755 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.909793 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.909785 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.911775 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.911751 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb86b57-5e55-4a1d-a9be-5f674fd80902-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.912049 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.912034 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb86b57-5e55-4a1d-a9be-5f674fd80902-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:24.918086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:24.918061 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pdp6\" (UniqueName: \"kubernetes.io/projected/efb86b57-5e55-4a1d-a9be-5f674fd80902-kube-api-access-9pdp6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jclmz\" (UID: \"efb86b57-5e55-4a1d-a9be-5f674fd80902\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:25.018592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:25.018496 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:25.291414 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:25.291382 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz"] Sep 08 15:27:25.293324 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:27:25.293292 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb86b57_5e55_4a1d_a9be_5f674fd80902.slice/crio-6a24cac5d9110f738716ab2eb949c7f361a6203e0d432e196187df2bd29d35bc WatchSource:0}: Error finding container 6a24cac5d9110f738716ab2eb949c7f361a6203e0d432e196187df2bd29d35bc: Status 404 returned error can't find the container with id 6a24cac5d9110f738716ab2eb949c7f361a6203e0d432e196187df2bd29d35bc Sep 08 15:27:25.697035 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:25.696997 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" event={"ID":"efb86b57-5e55-4a1d-a9be-5f674fd80902","Type":"ContainerStarted","Data":"6ea193daccf84a8b98c81db9f1ca7471a7b4707c7a6823db1a87bacff06c1214"} Sep 08 15:27:25.697219 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:25.697075 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" event={"ID":"efb86b57-5e55-4a1d-a9be-5f674fd80902","Type":"ContainerStarted","Data":"6a24cac5d9110f738716ab2eb949c7f361a6203e0d432e196187df2bd29d35bc"} Sep 08 15:27:25.752028 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:25.751973 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" podStartSLOduration=1.514558166 podStartE2EDuration="1.751956813s" totalImagesPullingTime="237.398647ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:25.750303676 +0000 UTC m=+584.773928133" watchObservedRunningTime="2026-09-08 15:27:25.751956813 +0000 UTC m=+584.775581268" Sep 08 15:27:27.730066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:27.730032 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:27.730480 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:27.730091 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:27.743038 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:27.743010 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:28.725074 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:28.725045 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75" Sep 08 15:27:35.018980 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:35.018932 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:35.019470 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:35.018998 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:35.032138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:35.032095 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:35.765358 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:35.765328 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jclmz" Sep 08 15:27:55.709130 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.709068 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw"] Sep 08 15:27:55.712500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.712478 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.717429 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.717402 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 08 15:27:55.728440 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.728405 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw"] Sep 08 15:27:55.802745 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.802705 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.802997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.802890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.802997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.802944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g84hs\" (UniqueName: \"kubernetes.io/projected/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-kube-api-access-g84hs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.802997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.802973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.803230 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.803061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.803230 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.803175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.903899 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.903857 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g84hs\" (UniqueName: \"kubernetes.io/projected/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-kube-api-access-g84hs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.904096 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.903913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.904096 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.903939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.904096 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.903970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.904096 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.904082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.904359 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.904196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.904465 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.904444 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.904551 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.904530 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.904602 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.904561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.906560 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.906529 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.907471 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.907448 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:55.912886 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:55.912861 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g84hs\" (UniqueName: \"kubernetes.io/projected/03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0-kube-api-access-g84hs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw\" (UID: \"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:56.022400 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:56.022297 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:27:56.309026 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:56.307781 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw"] Sep 08 15:27:56.309026 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:56.308711 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 15:27:56.877313 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:56.877206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" event={"ID":"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0","Type":"ContainerStarted","Data":"ba58f6d7d281061fd88dad1101ee5b42ff939d8d1c9240380ec523556a9705f4"} Sep 08 15:27:56.877313 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:56.877277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" event={"ID":"03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0","Type":"ContainerStarted","Data":"89b65f3a62a6a2145b53435e810078522153214e606f0f5f97bf8d28ce414574"} Sep 08 15:27:56.898752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:27:56.898697 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" podStartSLOduration=1.590834907 podStartE2EDuration="1.898681195s" totalImagesPullingTime="307.846288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:56.897704553 +0000 UTC m=+615.921329010" watchObservedRunningTime="2026-09-08 15:27:56.898681195 +0000 UTC m=+615.922305694" Sep 08 15:28:06.023500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:28:06.023408 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:28:06.023500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:28:06.023478 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:28:06.036593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:28:06.036557 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:28:06.948306 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:28:06.948276 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw" Sep 08 15:29:38.795941 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.795848 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-678545c99-wfv5g"] Sep 08 15:29:38.799617 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.799596 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678545c99-wfv5g" Sep 08 15:29:38.803702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.803680 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rvx67\"" Sep 08 15:29:38.803846 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.803740 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 08 15:29:38.812762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.812739 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-678545c99-wfv5g"] Sep 08 15:29:38.890463 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.890429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c456eae-1e57-45bd-95af-e546cfb0e52c-tls-cert\") pod \"authorino-678545c99-wfv5g\" (UID: \"7c456eae-1e57-45bd-95af-e546cfb0e52c\") " pod="kuadrant-system/authorino-678545c99-wfv5g" Sep 08 15:29:38.890641 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.890482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfgrk\" (UniqueName: \"kubernetes.io/projected/7c456eae-1e57-45bd-95af-e546cfb0e52c-kube-api-access-kfgrk\") pod \"authorino-678545c99-wfv5g\" (UID: \"7c456eae-1e57-45bd-95af-e546cfb0e52c\") " pod="kuadrant-system/authorino-678545c99-wfv5g" Sep 08 15:29:38.991449 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.991404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c456eae-1e57-45bd-95af-e546cfb0e52c-tls-cert\") pod \"authorino-678545c99-wfv5g\" (UID: \"7c456eae-1e57-45bd-95af-e546cfb0e52c\") " pod="kuadrant-system/authorino-678545c99-wfv5g" Sep 08 15:29:38.991628 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.991460 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfgrk\" (UniqueName: \"kubernetes.io/projected/7c456eae-1e57-45bd-95af-e546cfb0e52c-kube-api-access-kfgrk\") pod \"authorino-678545c99-wfv5g\" (UID: \"7c456eae-1e57-45bd-95af-e546cfb0e52c\") " pod="kuadrant-system/authorino-678545c99-wfv5g" Sep 08 15:29:38.994220 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:38.994186 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c456eae-1e57-45bd-95af-e546cfb0e52c-tls-cert\") pod \"authorino-678545c99-wfv5g\" (UID: \"7c456eae-1e57-45bd-95af-e546cfb0e52c\") " pod="kuadrant-system/authorino-678545c99-wfv5g" Sep 08 15:29:39.004262 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:39.004237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfgrk\" (UniqueName: \"kubernetes.io/projected/7c456eae-1e57-45bd-95af-e546cfb0e52c-kube-api-access-kfgrk\") pod \"authorino-678545c99-wfv5g\" (UID: \"7c456eae-1e57-45bd-95af-e546cfb0e52c\") " pod="kuadrant-system/authorino-678545c99-wfv5g" Sep 08 15:29:39.108538 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:39.108401 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678545c99-wfv5g" Sep 08 15:29:39.393954 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:39.393924 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-678545c99-wfv5g"] Sep 08 15:29:39.396280 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:29:39.396240 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c456eae_1e57_45bd_95af_e546cfb0e52c.slice/crio-ae888a9d616f1e3d2dd76d11a1fe28ae2a836228456bdad4da07e59715a3595e WatchSource:0}: Error finding container ae888a9d616f1e3d2dd76d11a1fe28ae2a836228456bdad4da07e59715a3595e: Status 404 returned error can't find the container with id ae888a9d616f1e3d2dd76d11a1fe28ae2a836228456bdad4da07e59715a3595e Sep 08 15:29:39.469495 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:39.469455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678545c99-wfv5g" event={"ID":"7c456eae-1e57-45bd-95af-e546cfb0e52c","Type":"ContainerStarted","Data":"ae888a9d616f1e3d2dd76d11a1fe28ae2a836228456bdad4da07e59715a3595e"} Sep 08 15:29:41.483311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:41.483265 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678545c99-wfv5g" event={"ID":"7c456eae-1e57-45bd-95af-e546cfb0e52c","Type":"ContainerStarted","Data":"e188f0f023361bb9cca15d7838ca52baeff26dad1982304929c38ed4ba8d962a"} Sep 08 15:29:41.514762 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:29:41.514692 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-678545c99-wfv5g" podStartSLOduration=2.026286247 podStartE2EDuration="3.51467199s" totalImagesPullingTime="1.488385743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:29:41.511196345 +0000 UTC m=+720.534820801" watchObservedRunningTime="2026-09-08 15:29:41.51467199 +0000 UTC m=+720.538296448" Sep 08 15:30:16.522562 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.522512 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk"] Sep 08 15:30:16.526337 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.526314 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.538842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.538815 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-425c6986-openshift-default-dockercfg-tx5rp\"" Sep 08 15:30:16.624826 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.624785 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk"] Sep 08 15:30:16.660418 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.660378 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-certs\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.660418 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.660416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-credential-socket\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.660651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.660518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-socket\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.660651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.660545 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istiod-ca-cert\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.660651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.660571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-podinfo\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.660651 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.660639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9rj5\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-kube-api-access-p9rj5\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.660802 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.660666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-data\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.660802 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.660692 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-token\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.660802 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.660712 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-envoy\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.761609 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.761561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-certs\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.761609 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.761610 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-credential-socket\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.761885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.761673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-socket\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.761885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.761689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istiod-ca-cert\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.761885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.761710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-podinfo\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.761885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.761747 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9rj5\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-kube-api-access-p9rj5\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.761885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.761779 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-data\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.761885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.761811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-token\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.761885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.761842 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-envoy\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.762297 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.762075 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-certs\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.762297 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.762174 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-socket\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.762402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.762291 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-credential-socket\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.762449 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.762407 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-data\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.762794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.762769 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istiod-ca-cert\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.764372 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.764352 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-envoy\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.764530 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.764510 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-podinfo\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.783950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.783864 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-token\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.794383 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.794347 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9rj5\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-kube-api-access-p9rj5\") pod \"e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:16.836494 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:16.836456 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:17.229003 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:17.228968 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk"] Sep 08 15:30:17.245058 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:30:17.245029 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd52b22_ba1b_4448_b9b2_307519e6d49b.slice/crio-30dbded2df97d362b65ba6741aebf6bde024edf4be2b38b345a8f270d4df28b6 WatchSource:0}: Error finding container 30dbded2df97d362b65ba6741aebf6bde024edf4be2b38b345a8f270d4df28b6: Status 404 returned error can't find the container with id 30dbded2df97d362b65ba6741aebf6bde024edf4be2b38b345a8f270d4df28b6 Sep 08 15:30:17.247328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:17.247293 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:30:17.247438 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:17.247378 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:30:17.247438 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:17.247422 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:30:17.702883 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:17.702850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" event={"ID":"bbd52b22-ba1b-4448-b9b2-307519e6d49b","Type":"ContainerStarted","Data":"3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2"} Sep 08 15:30:17.703281 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:17.702913 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" event={"ID":"bbd52b22-ba1b-4448-b9b2-307519e6d49b","Type":"ContainerStarted","Data":"30dbded2df97d362b65ba6741aebf6bde024edf4be2b38b345a8f270d4df28b6"} Sep 08 15:30:17.788660 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:17.788602 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" podStartSLOduration=1.7885825180000001 podStartE2EDuration="1.788582518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:17.78342373 +0000 UTC m=+756.807048199" watchObservedRunningTime="2026-09-08 15:30:17.788582518 +0000 UTC m=+756.812206973" Sep 08 15:30:17.837610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:17.837572 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:18.842528 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:18.842497 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:19.608238 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.608204 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs"] Sep 08 15:30:19.612026 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.611986 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:19.616369 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.616339 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-8debe9-dockercfg-4nj7f\"" Sep 08 15:30:19.616520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.616375 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-8debe9\"" Sep 08 15:30:19.693211 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.693174 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxnzm\" (UniqueName: \"kubernetes.io/projected/64fec928-5591-4dca-adbc-63aceb03544c-kube-api-access-wxnzm\") pod \"payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs\" (UID: \"64fec928-5591-4dca-adbc-63aceb03544c\") " pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:19.693457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.693379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64fec928-5591-4dca-adbc-63aceb03544c-plugins-config-volume\") pod \"payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs\" (UID: \"64fec928-5591-4dca-adbc-63aceb03544c\") " pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:19.715432 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.715401 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:19.716278 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.716254 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs"] Sep 08 15:30:19.716729 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.716708 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:19.794240 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.794204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64fec928-5591-4dca-adbc-63aceb03544c-plugins-config-volume\") pod \"payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs\" (UID: \"64fec928-5591-4dca-adbc-63aceb03544c\") " pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:19.794471 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.794256 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxnzm\" (UniqueName: \"kubernetes.io/projected/64fec928-5591-4dca-adbc-63aceb03544c-kube-api-access-wxnzm\") pod \"payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs\" (UID: \"64fec928-5591-4dca-adbc-63aceb03544c\") " pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:19.795015 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.794994 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64fec928-5591-4dca-adbc-63aceb03544c-plugins-config-volume\") pod \"payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs\" (UID: \"64fec928-5591-4dca-adbc-63aceb03544c\") " pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:19.863517 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.863438 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxnzm\" (UniqueName: \"kubernetes.io/projected/64fec928-5591-4dca-adbc-63aceb03544c-kube-api-access-wxnzm\") pod \"payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs\" (UID: \"64fec928-5591-4dca-adbc-63aceb03544c\") " pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:19.923763 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:19.923725 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:20.280715 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:30:20.280677 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64fec928_5591_4dca_adbc_63aceb03544c.slice/crio-b87b9422bff254d590f67dbd1f63cb2ceac985c608a49fdc66616ceac250f395 WatchSource:0}: Error finding container b87b9422bff254d590f67dbd1f63cb2ceac985c608a49fdc66616ceac250f395: Status 404 returned error can't find the container with id b87b9422bff254d590f67dbd1f63cb2ceac985c608a49fdc66616ceac250f395 Sep 08 15:30:20.284399 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:20.284376 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs"] Sep 08 15:30:20.722414 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:20.722370 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" event={"ID":"64fec928-5591-4dca-adbc-63aceb03544c","Type":"ContainerStarted","Data":"b87b9422bff254d590f67dbd1f63cb2ceac985c608a49fdc66616ceac250f395"} Sep 08 15:30:22.738758 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:22.738717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" event={"ID":"64fec928-5591-4dca-adbc-63aceb03544c","Type":"ContainerStarted","Data":"65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6"} Sep 08 15:30:22.739177 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:22.738789 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:22.788586 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:22.788520 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" podStartSLOduration=2.029290729 podStartE2EDuration="3.788499636s" totalImagesPullingTime="1.759208907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:22.785619306 +0000 UTC m=+761.809243761" watchObservedRunningTime="2026-09-08 15:30:22.788499636 +0000 UTC m=+761.812124092" Sep 08 15:30:33.748566 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:33.748528 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:30:38.839708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:38.839669 2578 generic.go:356] "Generic (PLEG): container finished" podID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerID="3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2" exitCode=0 Sep 08 15:30:38.840142 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:38.839745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" event={"ID":"bbd52b22-ba1b-4448-b9b2-307519e6d49b","Type":"ContainerDied","Data":"3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2"} Sep 08 15:30:38.840236 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:38.840222 2578 scope.go:117] "RemoveContainer" containerID="3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2" Sep 08 15:30:38.841521 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:38.841496 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:30:38.841587 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:38.841558 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:30:38.841625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:38.841594 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:30:39.852541 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:39.852491 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" event={"ID":"bbd52b22-ba1b-4448-b9b2-307519e6d49b","Type":"ContainerStarted","Data":"7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0"} Sep 08 15:30:40.837208 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:40.837163 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:40.837518 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:40.837487 2578 patch_prober.go:28] interesting pod/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Sep 08 15:30:40.837614 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:40.837572 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Sep 08 15:30:41.249590 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.249533 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg"] Sep 08 15:30:41.254182 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.254158 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.262323 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.262298 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e39762-openshift-default-dockercfg-g9s48\"" Sep 08 15:30:41.292215 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.292162 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg"] Sep 08 15:30:41.311372 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.311338 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz8m2\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-kube-api-access-qz8m2\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.311529 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.311386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7d9e082-b577-45de-b553-ba564c53c1b8-istiod-ca-cert\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.311583 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.311519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-envoy\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.311583 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.311579 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-socket\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.311662 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.311604 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-certs\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.311710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.311695 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-token\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.311769 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.311720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-credential-socket\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.311769 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.311741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-podinfo\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.311854 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.311771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-data\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412303 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-token\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412303 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-credential-socket\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412588 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-podinfo\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412588 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-data\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412588 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz8m2\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-kube-api-access-qz8m2\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412764 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412601 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7d9e082-b577-45de-b553-ba564c53c1b8-istiod-ca-cert\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412764 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412703 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-envoy\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412764 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412744 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-credential-socket\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412927 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412767 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-socket\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412927 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-certs\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.412927 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.412821 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-data\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.413168 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.413145 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-certs\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.413321 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.413272 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-socket\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.413442 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.413416 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7d9e082-b577-45de-b553-ba564c53c1b8-istiod-ca-cert\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.415470 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.415449 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-envoy\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.417127 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.417083 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-podinfo\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.470713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.470683 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz8m2\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-kube-api-access-qz8m2\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.533896 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.533747 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-token\") pod \"e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.570097 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.570067 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e39762-openshift-default-dockercfg-g9s48\"" Sep 08 15:30:41.575867 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.575838 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:41.839671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.839142 2578 patch_prober.go:28] interesting pod/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Sep 08 15:30:41.839671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.839220 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Sep 08 15:30:41.903900 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.903868 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg"] Sep 08 15:30:41.904878 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:30:41.904854 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d9e082_b577_45de_b553_ba564c53c1b8.slice/crio-76e2de02ad686aada44ae6237e0ef329dbb9afa86d24469688b06336e85e79e4 WatchSource:0}: Error finding container 76e2de02ad686aada44ae6237e0ef329dbb9afa86d24469688b06336e85e79e4: Status 404 returned error can't find the container with id 76e2de02ad686aada44ae6237e0ef329dbb9afa86d24469688b06336e85e79e4 Sep 08 15:30:41.907249 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.907222 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:30:41.907308 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.907285 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:30:41.907349 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:41.907316 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:30:42.837617 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:42.837135 2578 patch_prober.go:28] interesting pod/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Sep 08 15:30:42.837617 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:42.837225 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Sep 08 15:30:42.873008 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:42.872974 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" event={"ID":"d7d9e082-b577-45de-b553-ba564c53c1b8","Type":"ContainerStarted","Data":"af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282"} Sep 08 15:30:42.873201 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:42.873052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" event={"ID":"d7d9e082-b577-45de-b553-ba564c53c1b8","Type":"ContainerStarted","Data":"76e2de02ad686aada44ae6237e0ef329dbb9afa86d24469688b06336e85e79e4"} Sep 08 15:30:42.972697 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:42.972637 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" podStartSLOduration=1.972619421 podStartE2EDuration="1.972619421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:42.970249585 +0000 UTC m=+781.993874053" watchObservedRunningTime="2026-09-08 15:30:42.972619421 +0000 UTC m=+781.996243878" Sep 08 15:30:43.576396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:43.576360 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:43.581396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:43.581370 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:43.837157 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:43.837032 2578 patch_prober.go:28] interesting pod/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Sep 08 15:30:43.837157 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:43.837133 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Sep 08 15:30:43.879949 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:43.879916 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:43.880915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:43.880894 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:30:44.837379 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:44.837338 2578 patch_prober.go:28] interesting pod/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Sep 08 15:30:44.837659 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:44.837631 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Sep 08 15:30:45.840976 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:45.840946 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:45.841380 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:45.841311 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:45.841994 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:45.841975 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:30:50.161731 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.161692 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2"] Sep 08 15:30:50.168775 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.168743 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" Sep 08 15:30:50.198766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.198735 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:30:50.199004 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.198989 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:30:50.214839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.214804 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36934154-6295-499c-9730-256a2aeab678-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-425c6986-89fr2\" (UID: \"36934154-6295-499c-9730-256a2aeab678\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" Sep 08 15:30:50.232688 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.232658 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2"] Sep 08 15:30:50.315683 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.315642 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36934154-6295-499c-9730-256a2aeab678-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-425c6986-89fr2\" (UID: \"36934154-6295-499c-9730-256a2aeab678\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" Sep 08 15:30:50.316275 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.316256 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36934154-6295-499c-9730-256a2aeab678-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-425c6986-89fr2\" (UID: \"36934154-6295-499c-9730-256a2aeab678\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" Sep 08 15:30:50.482435 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.482393 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" Sep 08 15:30:50.807024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.806994 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2"] Sep 08 15:30:50.807675 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:30:50.807650 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36934154_6295_499c_9730_256a2aeab678.slice/crio-4d8933ee25ec809f0b9cee55d7725aa6b3124a89c346419822f381c242578884 WatchSource:0}: Error finding container 4d8933ee25ec809f0b9cee55d7725aa6b3124a89c346419822f381c242578884: Status 404 returned error can't find the container with id 4d8933ee25ec809f0b9cee55d7725aa6b3124a89c346419822f381c242578884 Sep 08 15:30:50.931595 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:50.931563 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" event={"ID":"36934154-6295-499c-9730-256a2aeab678","Type":"ContainerStarted","Data":"4d8933ee25ec809f0b9cee55d7725aa6b3124a89c346419822f381c242578884"} Sep 08 15:30:53.953070 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:53.953029 2578 generic.go:356] "Generic (PLEG): container finished" podID="36934154-6295-499c-9730-256a2aeab678" containerID="ea7418a5bbcf58eda5e68d3a483f45a8c89c0e0930ab73f0c3716a350645a392" exitCode=0 Sep 08 15:30:53.953488 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:53.953087 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" event={"ID":"36934154-6295-499c-9730-256a2aeab678","Type":"ContainerDied","Data":"ea7418a5bbcf58eda5e68d3a483f45a8c89c0e0930ab73f0c3716a350645a392"} Sep 08 15:30:55.103475 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:55.103449 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" Sep 08 15:30:55.161198 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:55.161154 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36934154-6295-499c-9730-256a2aeab678-maas-api-service-ca\") pod \"36934154-6295-499c-9730-256a2aeab678\" (UID: \"36934154-6295-499c-9730-256a2aeab678\") " Sep 08 15:30:55.161576 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:55.161553 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36934154-6295-499c-9730-256a2aeab678-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "36934154-6295-499c-9730-256a2aeab678" (UID: "36934154-6295-499c-9730-256a2aeab678"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:30:55.262469 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:55.262368 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36934154-6295-499c-9730-256a2aeab678-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:30:55.965705 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:55.965672 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" Sep 08 15:30:55.965877 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:55.965704 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2" event={"ID":"36934154-6295-499c-9730-256a2aeab678","Type":"ContainerDied","Data":"4d8933ee25ec809f0b9cee55d7725aa6b3124a89c346419822f381c242578884"} Sep 08 15:30:55.965877 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:30:55.965754 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d8933ee25ec809f0b9cee55d7725aa6b3124a89c346419822f381c242578884" Sep 08 15:31:03.276017 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:03.275931 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2"] Sep 08 15:31:03.278471 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:03.278443 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-425c6986-89fr2"] Sep 08 15:31:03.447138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:03.446931 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk"] Sep 08 15:31:03.447325 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:03.447238 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" containerID="cri-o://7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0" gracePeriod=30 Sep 08 15:31:03.501810 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:03.501777 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36934154-6295-499c-9730-256a2aeab678" path="/var/lib/kubelet/pods/36934154-6295-499c-9730-256a2aeab678/volumes" Sep 08 15:31:04.558530 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.558447 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5"] Sep 08 15:31:04.558906 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.558892 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36934154-6295-499c-9730-256a2aeab678" containerName="revoke-keys" Sep 08 15:31:04.558949 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.558908 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="36934154-6295-499c-9730-256a2aeab678" containerName="revoke-keys" Sep 08 15:31:04.559016 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.559007 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="36934154-6295-499c-9730-256a2aeab678" containerName="revoke-keys" Sep 08 15:31:04.562348 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.562325 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.566462 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.566429 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-96c53ecc-gw-openshift-default-dockercfg-fb95r\"" Sep 08 15:31:04.580927 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.580895 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5"] Sep 08 15:31:04.679713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.679665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7db7c52-912e-49ed-bd76-51518449ca1f-istiod-ca-cert\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.679891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.679786 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-data\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.679891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.679839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-certs\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.679963 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.679931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-token\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.679998 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.679960 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx4h7\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-kube-api-access-rx4h7\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.680034 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.680001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-podinfo\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.680034 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.680026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-socket\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.680145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.680053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-credential-socket\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.680145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.680092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-envoy\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781486 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781453 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-data\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-certs\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-token\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx4h7\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-kube-api-access-rx4h7\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-podinfo\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-socket\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781938 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781690 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-credential-socket\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781938 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-envoy\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781938 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781762 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7db7c52-912e-49ed-bd76-51518449ca1f-istiod-ca-cert\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.781938 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.781924 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-data\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.782157 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.782005 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-certs\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.782157 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.782021 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-credential-socket\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.782332 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.782305 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-socket\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.782461 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.782411 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7db7c52-912e-49ed-bd76-51518449ca1f-istiod-ca-cert\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.784158 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.784133 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-envoy\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.784503 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.784481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-podinfo\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.800573 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.800538 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-token\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.800798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.800780 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx4h7\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-kube-api-access-rx4h7\") pod \"e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:04.873820 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:04.873717 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:05.186187 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:05.186149 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5"] Sep 08 15:31:05.187020 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:31:05.186995 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7db7c52_912e_49ed_bd76_51518449ca1f.slice/crio-7e4ff0769ca3c2c22a7e0ae684642fbbd9119672898723bcb56b0b82bf72be6c WatchSource:0}: Error finding container 7e4ff0769ca3c2c22a7e0ae684642fbbd9119672898723bcb56b0b82bf72be6c: Status 404 returned error can't find the container with id 7e4ff0769ca3c2c22a7e0ae684642fbbd9119672898723bcb56b0b82bf72be6c Sep 08 15:31:05.189315 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:05.189281 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:05.189413 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:05.189347 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:05.189413 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:05.189378 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:06.030411 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:06.030368 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" event={"ID":"b7db7c52-912e-49ed-bd76-51518449ca1f","Type":"ContainerStarted","Data":"d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac"} Sep 08 15:31:06.030847 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:06.030470 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" event={"ID":"b7db7c52-912e-49ed-bd76-51518449ca1f","Type":"ContainerStarted","Data":"7e4ff0769ca3c2c22a7e0ae684642fbbd9119672898723bcb56b0b82bf72be6c"} Sep 08 15:31:06.058630 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:06.058560 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" podStartSLOduration=2.058529482 podStartE2EDuration="2.058529482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:06.057944505 +0000 UTC m=+805.081568961" watchObservedRunningTime="2026-09-08 15:31:06.058529482 +0000 UTC m=+805.082153941" Sep 08 15:31:06.874354 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:06.874309 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:06.880332 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:06.880296 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:07.039088 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:07.039050 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:07.040370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:07.040347 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:08.047105 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.047070 2578 generic.go:356] "Generic (PLEG): container finished" podID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerID="af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282" exitCode=0 Sep 08 15:31:08.047646 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.047166 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" event={"ID":"d7d9e082-b577-45de-b553-ba564c53c1b8","Type":"ContainerDied","Data":"af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282"} Sep 08 15:31:08.047785 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.047767 2578 scope.go:117] "RemoveContainer" containerID="af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282" Sep 08 15:31:08.048859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.048832 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:08.048950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.048909 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:08.048997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.048948 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:08.727172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.727037 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:31:08.828201 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828166 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-credential-socket\") pod \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " Sep 08 15:31:08.828402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828217 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-socket\") pod \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " Sep 08 15:31:08.828402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828255 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-certs\") pod \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " Sep 08 15:31:08.828402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828283 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istiod-ca-cert\") pod \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " Sep 08 15:31:08.828402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828333 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-podinfo\") pod \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " Sep 08 15:31:08.828402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828363 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9rj5\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-kube-api-access-p9rj5\") pod \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " Sep 08 15:31:08.828681 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828414 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-data\") pod \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " Sep 08 15:31:08.828681 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828503 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bbd52b22-ba1b-4448-b9b2-307519e6d49b" (UID: "bbd52b22-ba1b-4448-b9b2-307519e6d49b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:08.828681 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828522 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bbd52b22-ba1b-4448-b9b2-307519e6d49b" (UID: "bbd52b22-ba1b-4448-b9b2-307519e6d49b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:08.828836 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828696 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bbd52b22-ba1b-4448-b9b2-307519e6d49b" (UID: "bbd52b22-ba1b-4448-b9b2-307519e6d49b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:31:08.828836 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828713 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-data" (OuterVolumeSpecName: "istio-data") pod "bbd52b22-ba1b-4448-b9b2-307519e6d49b" (UID: "bbd52b22-ba1b-4448-b9b2-307519e6d49b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:08.828956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828837 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bbd52b22-ba1b-4448-b9b2-307519e6d49b" (UID: "bbd52b22-ba1b-4448-b9b2-307519e6d49b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:08.828956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828843 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:08.828956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828885 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:08.828956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828901 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:08.828956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.828915 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:08.830799 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.830776 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-kube-api-access-p9rj5" (OuterVolumeSpecName: "kube-api-access-p9rj5") pod "bbd52b22-ba1b-4448-b9b2-307519e6d49b" (UID: "bbd52b22-ba1b-4448-b9b2-307519e6d49b"). InnerVolumeSpecName "kube-api-access-p9rj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:31:08.831032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.831013 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bbd52b22-ba1b-4448-b9b2-307519e6d49b" (UID: "bbd52b22-ba1b-4448-b9b2-307519e6d49b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:31:08.929543 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.929505 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-token\") pod \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " Sep 08 15:31:08.929543 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.929557 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-envoy\") pod \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\" (UID: \"bbd52b22-ba1b-4448-b9b2-307519e6d49b\") " Sep 08 15:31:08.929849 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.929834 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:08.929849 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.929848 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:08.929968 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.929857 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9rj5\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-kube-api-access-p9rj5\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:08.931933 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.931899 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bbd52b22-ba1b-4448-b9b2-307519e6d49b" (UID: "bbd52b22-ba1b-4448-b9b2-307519e6d49b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:08.932070 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:08.931932 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-token" (OuterVolumeSpecName: "istio-token") pod "bbd52b22-ba1b-4448-b9b2-307519e6d49b" (UID: "bbd52b22-ba1b-4448-b9b2-307519e6d49b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:31:09.031175 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.031076 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:09.031175 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.031139 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd52b22-ba1b-4448-b9b2-307519e6d49b-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:09.057180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.057146 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" event={"ID":"d7d9e082-b577-45de-b553-ba564c53c1b8","Type":"ContainerStarted","Data":"b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a"} Sep 08 15:31:09.059170 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.059141 2578 generic.go:356] "Generic (PLEG): container finished" podID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerID="7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0" exitCode=0 Sep 08 15:31:09.059300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.059192 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" Sep 08 15:31:09.059300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.059233 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" event={"ID":"bbd52b22-ba1b-4448-b9b2-307519e6d49b","Type":"ContainerDied","Data":"7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0"} Sep 08 15:31:09.059404 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.059295 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk" event={"ID":"bbd52b22-ba1b-4448-b9b2-307519e6d49b","Type":"ContainerDied","Data":"30dbded2df97d362b65ba6741aebf6bde024edf4be2b38b345a8f270d4df28b6"} Sep 08 15:31:09.059404 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.059328 2578 scope.go:117] "RemoveContainer" containerID="7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0" Sep 08 15:31:09.069601 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.069578 2578 scope.go:117] "RemoveContainer" containerID="3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2" Sep 08 15:31:09.079241 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.079220 2578 scope.go:117] "RemoveContainer" containerID="7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0" Sep 08 15:31:09.079591 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:31:09.079568 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0\": container with ID starting with 7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0 not found: ID does not exist" containerID="7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0" Sep 08 15:31:09.079664 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.079603 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0"} err="failed to get container status \"7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0\": rpc error: code = NotFound desc = could not find container \"7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0\": container with ID starting with 7194cf8335703f8d731c7edeccff03ffc3ac7f7069f32f29774690a6dd3911c0 not found: ID does not exist" Sep 08 15:31:09.079664 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.079621 2578 scope.go:117] "RemoveContainer" containerID="3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2" Sep 08 15:31:09.079895 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:31:09.079880 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2\": container with ID starting with 3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2 not found: ID does not exist" containerID="3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2" Sep 08 15:31:09.079936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.079898 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2"} err="failed to get container status \"3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2\": rpc error: code = NotFound desc = could not find container \"3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2\": container with ID starting with 3d3800573f6119c96a6f085269aea4bef8acb09a4c72d9d6277b469b834e7bc2 not found: ID does not exist" Sep 08 15:31:09.109925 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.109891 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk"] Sep 08 15:31:09.119542 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.119510 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-425c6986-openshift-default-5674bb7d7d-wztrk"] Sep 08 15:31:09.503647 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.503607 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" path="/var/lib/kubelet/pods/bbd52b22-ba1b-4448-b9b2-307519e6d49b/volumes" Sep 08 15:31:09.576434 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.576394 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:31:09.577764 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.577736 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Sep 08 15:31:09.577903 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:09.577807 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Sep 08 15:31:10.577099 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:10.577064 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Sep 08 15:31:10.577484 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:10.577156 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Sep 08 15:31:11.577289 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:11.577248 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Sep 08 15:31:11.577717 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:11.577308 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Sep 08 15:31:12.576808 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:12.576764 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Sep 08 15:31:12.576994 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:12.576824 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Sep 08 15:31:13.577238 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:13.577190 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Sep 08 15:31:13.577661 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:13.577265 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Sep 08 15:31:13.879960 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:13.879868 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:31:14.577004 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:14.576965 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Sep 08 15:31:14.577211 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:14.577024 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Sep 08 15:31:15.580720 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:15.580685 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:31:15.581913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:15.581892 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:31:28.202963 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.202923 2578 generic.go:356] "Generic (PLEG): container finished" podID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerID="d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac" exitCode=0 Sep 08 15:31:28.202963 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.202963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" event={"ID":"b7db7c52-912e-49ed-bd76-51518449ca1f","Type":"ContainerDied","Data":"d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac"} Sep 08 15:31:28.203434 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.203405 2578 scope.go:117] "RemoveContainer" containerID="d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac" Sep 08 15:31:28.204522 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.204496 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:28.204585 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.204571 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:28.204621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.204605 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:28.492903 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.492803 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6"] Sep 08 15:31:28.493261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.493245 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" Sep 08 15:31:28.493261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.493260 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" Sep 08 15:31:28.493423 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.493348 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" Sep 08 15:31:28.493423 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.493359 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" Sep 08 15:31:28.496706 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.496683 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" Sep 08 15:31:28.500046 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.500023 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:31:28.500187 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.500024 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:31:28.508620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.508594 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6"] Sep 08 15:31:28.632530 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.632494 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a211fbe-04e5-4c62-9436-0e7d0f6a102d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8debe9-4hnb6\" (UID: \"5a211fbe-04e5-4c62-9436-0e7d0f6a102d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" Sep 08 15:31:28.734657 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.733572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a211fbe-04e5-4c62-9436-0e7d0f6a102d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8debe9-4hnb6\" (UID: \"5a211fbe-04e5-4c62-9436-0e7d0f6a102d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" Sep 08 15:31:28.734657 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.734597 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a211fbe-04e5-4c62-9436-0e7d0f6a102d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8debe9-4hnb6\" (UID: \"5a211fbe-04e5-4c62-9436-0e7d0f6a102d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" Sep 08 15:31:28.809478 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:28.809387 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" Sep 08 15:31:29.121453 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:29.121426 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6"] Sep 08 15:31:29.123576 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:31:29.123517 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a211fbe_04e5_4c62_9436_0e7d0f6a102d.slice/crio-f88d0ec59b15a62e5485407b85842f787bae65b3ede86eded695f237337608f9 WatchSource:0}: Error finding container f88d0ec59b15a62e5485407b85842f787bae65b3ede86eded695f237337608f9: Status 404 returned error can't find the container with id f88d0ec59b15a62e5485407b85842f787bae65b3ede86eded695f237337608f9 Sep 08 15:31:29.211413 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:29.211378 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" event={"ID":"b7db7c52-912e-49ed-bd76-51518449ca1f","Type":"ContainerStarted","Data":"d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c"} Sep 08 15:31:29.212682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:29.212659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" event={"ID":"5a211fbe-04e5-4c62-9436-0e7d0f6a102d","Type":"ContainerStarted","Data":"f88d0ec59b15a62e5485407b85842f787bae65b3ede86eded695f237337608f9"} Sep 08 15:31:29.874505 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:29.874417 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:29.874764 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:29.874740 2578 patch_prober.go:28] interesting pod/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 08 15:31:29.874811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:29.874792 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 08 15:31:30.220301 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:30.220264 2578 generic.go:356] "Generic (PLEG): container finished" podID="5a211fbe-04e5-4c62-9436-0e7d0f6a102d" containerID="f7ec2f3463fe308016f0918b1e872f302916de770bc852ef7cf91d9f2c69f7d8" exitCode=0 Sep 08 15:31:30.220748 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:30.220350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" event={"ID":"5a211fbe-04e5-4c62-9436-0e7d0f6a102d","Type":"ContainerDied","Data":"f7ec2f3463fe308016f0918b1e872f302916de770bc852ef7cf91d9f2c69f7d8"} Sep 08 15:31:30.874412 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:30.874369 2578 patch_prober.go:28] interesting pod/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 08 15:31:30.874614 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:30.874433 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 08 15:31:31.399699 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:31.399673 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" Sep 08 15:31:31.471600 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:31.471559 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a211fbe-04e5-4c62-9436-0e7d0f6a102d-maas-api-service-ca\") pod \"5a211fbe-04e5-4c62-9436-0e7d0f6a102d\" (UID: \"5a211fbe-04e5-4c62-9436-0e7d0f6a102d\") " Sep 08 15:31:31.471998 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:31.471971 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a211fbe-04e5-4c62-9436-0e7d0f6a102d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5a211fbe-04e5-4c62-9436-0e7d0f6a102d" (UID: "5a211fbe-04e5-4c62-9436-0e7d0f6a102d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:31:31.572672 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:31.572628 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a211fbe-04e5-4c62-9436-0e7d0f6a102d-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:31.874865 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:31.874770 2578 patch_prober.go:28] interesting pod/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 08 15:31:31.874865 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:31.874840 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 08 15:31:32.242582 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.242503 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" event={"ID":"5a211fbe-04e5-4c62-9436-0e7d0f6a102d","Type":"ContainerDied","Data":"f88d0ec59b15a62e5485407b85842f787bae65b3ede86eded695f237337608f9"} Sep 08 15:31:32.242798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.242609 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f88d0ec59b15a62e5485407b85842f787bae65b3ede86eded695f237337608f9" Sep 08 15:31:32.242798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.242684 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6" Sep 08 15:31:32.493234 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.493146 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567"] Sep 08 15:31:32.493589 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.493576 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a211fbe-04e5-4c62-9436-0e7d0f6a102d" containerName="revoke-keys" Sep 08 15:31:32.493630 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.493589 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a211fbe-04e5-4c62-9436-0e7d0f6a102d" containerName="revoke-keys" Sep 08 15:31:32.493630 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.493600 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" Sep 08 15:31:32.493630 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.493605 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd52b22-ba1b-4448-b9b2-307519e6d49b" containerName="istio-proxy" Sep 08 15:31:32.493725 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.493685 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a211fbe-04e5-4c62-9436-0e7d0f6a102d" containerName="revoke-keys" Sep 08 15:31:32.496969 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.496940 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" Sep 08 15:31:32.502409 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.502390 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:31:32.507564 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.507545 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:31:32.521833 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.521798 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567"] Sep 08 15:31:32.582982 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.582926 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab88a648-51b8-4856-ab45-78f8440d5835-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567\" (UID: \"ab88a648-51b8-4856-ab45-78f8440d5835\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" Sep 08 15:31:32.684333 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.684293 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab88a648-51b8-4856-ab45-78f8440d5835-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567\" (UID: \"ab88a648-51b8-4856-ab45-78f8440d5835\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" Sep 08 15:31:32.684928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.684904 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab88a648-51b8-4856-ab45-78f8440d5835-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567\" (UID: \"ab88a648-51b8-4856-ab45-78f8440d5835\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" Sep 08 15:31:32.806572 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.806475 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" Sep 08 15:31:32.874936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.874849 2578 patch_prober.go:28] interesting pod/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 08 15:31:32.874936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:32.874920 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 08 15:31:33.149180 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:31:33.140535 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab88a648_51b8_4856_ab45_78f8440d5835.slice/crio-76742a4475d2b2611491b1a04d41429366850a8432aa4500ccfda7ac51b1aab4 WatchSource:0}: Error finding container 76742a4475d2b2611491b1a04d41429366850a8432aa4500ccfda7ac51b1aab4: Status 404 returned error can't find the container with id 76742a4475d2b2611491b1a04d41429366850a8432aa4500ccfda7ac51b1aab4 Sep 08 15:31:33.149180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:33.140731 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567"] Sep 08 15:31:33.258095 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:33.258062 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" event={"ID":"ab88a648-51b8-4856-ab45-78f8440d5835","Type":"ContainerStarted","Data":"76742a4475d2b2611491b1a04d41429366850a8432aa4500ccfda7ac51b1aab4"} Sep 08 15:31:33.874434 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:33.874340 2578 patch_prober.go:28] interesting pod/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 08 15:31:33.874434 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:33.874395 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 08 15:31:34.266498 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:34.266457 2578 generic.go:356] "Generic (PLEG): container finished" podID="ab88a648-51b8-4856-ab45-78f8440d5835" containerID="4f7af3df88424e2b807eaff7b663af1d15cab4866ca886e6e1f42b5351b21e53" exitCode=0 Sep 08 15:31:34.266710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:34.266550 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" event={"ID":"ab88a648-51b8-4856-ab45-78f8440d5835","Type":"ContainerDied","Data":"4f7af3df88424e2b807eaff7b663af1d15cab4866ca886e6e1f42b5351b21e53"} Sep 08 15:31:34.878701 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:34.878670 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:34.879169 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:34.878962 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:34.879663 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:34.879643 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:35.418088 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:35.418063 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" Sep 08 15:31:35.513456 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:35.513412 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab88a648-51b8-4856-ab45-78f8440d5835-maas-api-service-ca\") pod \"ab88a648-51b8-4856-ab45-78f8440d5835\" (UID: \"ab88a648-51b8-4856-ab45-78f8440d5835\") " Sep 08 15:31:35.513807 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:35.513782 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab88a648-51b8-4856-ab45-78f8440d5835-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab88a648-51b8-4856-ab45-78f8440d5835" (UID: "ab88a648-51b8-4856-ab45-78f8440d5835"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:31:35.615072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:35.614975 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab88a648-51b8-4856-ab45-78f8440d5835-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:36.280161 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:36.280126 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" event={"ID":"ab88a648-51b8-4856-ab45-78f8440d5835","Type":"ContainerDied","Data":"76742a4475d2b2611491b1a04d41429366850a8432aa4500ccfda7ac51b1aab4"} Sep 08 15:31:36.280634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:36.280179 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567" Sep 08 15:31:36.280634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:36.280184 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76742a4475d2b2611491b1a04d41429366850a8432aa4500ccfda7ac51b1aab4" Sep 08 15:31:40.034354 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.034315 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs"] Sep 08 15:31:40.036183 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.034963 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" podUID="64fec928-5591-4dca-adbc-63aceb03544c" containerName="payload-processing" containerID="cri-o://65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6" gracePeriod=30 Sep 08 15:31:40.124741 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.124694 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567"] Sep 08 15:31:40.134200 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.134162 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-96c53ecc-2k567"] Sep 08 15:31:40.269979 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.269937 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5"] Sep 08 15:31:40.270250 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.270207 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" containerID="cri-o://d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c" gracePeriod=30 Sep 08 15:31:40.304630 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.304602 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:31:40.306517 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.306489 2578 generic.go:356] "Generic (PLEG): container finished" podID="64fec928-5591-4dca-adbc-63aceb03544c" containerID="65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6" exitCode=0 Sep 08 15:31:40.306668 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.306544 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" Sep 08 15:31:40.306668 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.306575 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" event={"ID":"64fec928-5591-4dca-adbc-63aceb03544c","Type":"ContainerDied","Data":"65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6"} Sep 08 15:31:40.306668 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.306637 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs" event={"ID":"64fec928-5591-4dca-adbc-63aceb03544c","Type":"ContainerDied","Data":"b87b9422bff254d590f67dbd1f63cb2ceac985c608a49fdc66616ceac250f395"} Sep 08 15:31:40.306668 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.306662 2578 scope.go:117] "RemoveContainer" containerID="65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6" Sep 08 15:31:40.316147 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.316095 2578 scope.go:117] "RemoveContainer" containerID="65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6" Sep 08 15:31:40.316481 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:31:40.316458 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6\": container with ID starting with 65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6 not found: ID does not exist" containerID="65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6" Sep 08 15:31:40.316552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.316491 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6"} err="failed to get container status \"65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6\": rpc error: code = NotFound desc = could not find container \"65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6\": container with ID starting with 65441264188cfb78455e0c3aab5c6d6d4d53aba2783f549b0c0eb18a727bfca6 not found: ID does not exist" Sep 08 15:31:40.363748 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.363710 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64fec928-5591-4dca-adbc-63aceb03544c-plugins-config-volume\") pod \"64fec928-5591-4dca-adbc-63aceb03544c\" (UID: \"64fec928-5591-4dca-adbc-63aceb03544c\") " Sep 08 15:31:40.363920 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.363805 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxnzm\" (UniqueName: \"kubernetes.io/projected/64fec928-5591-4dca-adbc-63aceb03544c-kube-api-access-wxnzm\") pod \"64fec928-5591-4dca-adbc-63aceb03544c\" (UID: \"64fec928-5591-4dca-adbc-63aceb03544c\") " Sep 08 15:31:40.364168 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.364129 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64fec928-5591-4dca-adbc-63aceb03544c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64fec928-5591-4dca-adbc-63aceb03544c" (UID: "64fec928-5591-4dca-adbc-63aceb03544c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:31:40.366197 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.366171 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64fec928-5591-4dca-adbc-63aceb03544c-kube-api-access-wxnzm" (OuterVolumeSpecName: "kube-api-access-wxnzm") pod "64fec928-5591-4dca-adbc-63aceb03544c" (UID: "64fec928-5591-4dca-adbc-63aceb03544c"). InnerVolumeSpecName "kube-api-access-wxnzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:31:40.465006 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.464964 2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64fec928-5591-4dca-adbc-63aceb03544c-plugins-config-volume\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:40.465006 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.464998 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxnzm\" (UniqueName: \"kubernetes.io/projected/64fec928-5591-4dca-adbc-63aceb03544c-kube-api-access-wxnzm\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:40.603608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.603507 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6"] Sep 08 15:31:40.613193 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.613163 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8debe9-4hnb6"] Sep 08 15:31:40.648317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.648277 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs"] Sep 08 15:31:40.654260 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:40.654232 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8debe9-75576b4b59-f8lfs"] Sep 08 15:31:41.204369 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.204322 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg"] Sep 08 15:31:41.204812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.204798 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab88a648-51b8-4856-ab45-78f8440d5835" containerName="revoke-keys" Sep 08 15:31:41.204812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.204813 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab88a648-51b8-4856-ab45-78f8440d5835" containerName="revoke-keys" Sep 08 15:31:41.204899 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.204843 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64fec928-5591-4dca-adbc-63aceb03544c" containerName="payload-processing" Sep 08 15:31:41.204899 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.204849 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64fec928-5591-4dca-adbc-63aceb03544c" containerName="payload-processing" Sep 08 15:31:41.204962 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.204918 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab88a648-51b8-4856-ab45-78f8440d5835" containerName="revoke-keys" Sep 08 15:31:41.204962 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.204929 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="64fec928-5591-4dca-adbc-63aceb03544c" containerName="payload-processing" Sep 08 15:31:41.209426 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.209403 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.219439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.219417 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f86ebb1f-openshift-default-dockercfg-8x4fv\"" Sep 08 15:31:41.237967 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.237931 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg"] Sep 08 15:31:41.274653 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.274619 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-credential-socket\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.274821 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.274660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-podinfo\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.274821 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.274682 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db4jb\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-kube-api-access-db4jb\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.274821 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.274765 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-token\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.274821 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.274818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f17a0c7-c290-48cb-83d3-f2d781174c58-istiod-ca-cert\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.274957 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.274872 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-data\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.274957 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.274902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-socket\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.274957 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.274948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-envoy\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.275060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.274962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-certs\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.375836 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.375795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f17a0c7-c290-48cb-83d3-f2d781174c58-istiod-ca-cert\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.375848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-data\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.375879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-socket\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.375911 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-envoy\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.375926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-certs\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.375976 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-credential-socket\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.375997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-podinfo\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.376013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db4jb\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-kube-api-access-db4jb\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376508 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.376042 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-token\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376508 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.376324 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-socket\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376508 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.376378 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-data\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376645 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.376611 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f17a0c7-c290-48cb-83d3-f2d781174c58-istiod-ca-cert\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376695 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.376673 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-credential-socket\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.376695 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.376687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-certs\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.378591 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.378567 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-envoy\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.378724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.378705 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-podinfo\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.388387 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.388355 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-token\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.388387 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.388377 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db4jb\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-kube-api-access-db4jb\") pod \"e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.502739 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.502659 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a211fbe-04e5-4c62-9436-0e7d0f6a102d" path="/var/lib/kubelet/pods/5a211fbe-04e5-4c62-9436-0e7d0f6a102d/volumes" Sep 08 15:31:41.503039 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.503026 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64fec928-5591-4dca-adbc-63aceb03544c" path="/var/lib/kubelet/pods/64fec928-5591-4dca-adbc-63aceb03544c/volumes" Sep 08 15:31:41.503357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.503344 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab88a648-51b8-4856-ab45-78f8440d5835" path="/var/lib/kubelet/pods/ab88a648-51b8-4856-ab45-78f8440d5835/volumes" Sep 08 15:31:41.524609 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.524581 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f86ebb1f-openshift-default-dockercfg-8x4fv\"" Sep 08 15:31:41.532030 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.532008 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:41.831764 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:31:41.831732 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f17a0c7_c290_48cb_83d3_f2d781174c58.slice/crio-5e0dec7c38cb8fd544f22ecee64d522a209720e2343c17a1b008db5c034d341f WatchSource:0}: Error finding container 5e0dec7c38cb8fd544f22ecee64d522a209720e2343c17a1b008db5c034d341f: Status 404 returned error can't find the container with id 5e0dec7c38cb8fd544f22ecee64d522a209720e2343c17a1b008db5c034d341f Sep 08 15:31:41.833406 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.833288 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg"] Sep 08 15:31:41.833986 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.833951 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:41.834076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.834033 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:41.834152 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:41.834075 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:42.325027 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:42.324987 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" event={"ID":"8f17a0c7-c290-48cb-83d3-f2d781174c58","Type":"ContainerStarted","Data":"eef75bc94c5f144269d3174d129a80f38bc0f42fd58bbb491cb37d35325fcfd2"} Sep 08 15:31:42.325524 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:42.325042 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" event={"ID":"8f17a0c7-c290-48cb-83d3-f2d781174c58","Type":"ContainerStarted","Data":"5e0dec7c38cb8fd544f22ecee64d522a209720e2343c17a1b008db5c034d341f"} Sep 08 15:31:42.381888 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:42.381824 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" podStartSLOduration=1.381800594 podStartE2EDuration="1.381800594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:42.376323266 +0000 UTC m=+841.399947722" watchObservedRunningTime="2026-09-08 15:31:42.381800594 +0000 UTC m=+841.405425051" Sep 08 15:31:42.532590 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:42.532545 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:42.537538 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:42.537511 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:43.330935 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:43.330898 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:43.331980 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:43.331960 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:31:45.542576 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.542502 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:45.624343 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624297 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-envoy\") pod \"b7db7c52-912e-49ed-bd76-51518449ca1f\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " Sep 08 15:31:45.624343 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624348 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-token\") pod \"b7db7c52-912e-49ed-bd76-51518449ca1f\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " Sep 08 15:31:45.624593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624423 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-socket\") pod \"b7db7c52-912e-49ed-bd76-51518449ca1f\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " Sep 08 15:31:45.624593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624468 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7db7c52-912e-49ed-bd76-51518449ca1f-istiod-ca-cert\") pod \"b7db7c52-912e-49ed-bd76-51518449ca1f\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " Sep 08 15:31:45.624593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624535 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-certs\") pod \"b7db7c52-912e-49ed-bd76-51518449ca1f\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " Sep 08 15:31:45.624593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624578 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-credential-socket\") pod \"b7db7c52-912e-49ed-bd76-51518449ca1f\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " Sep 08 15:31:45.624794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624604 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx4h7\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-kube-api-access-rx4h7\") pod \"b7db7c52-912e-49ed-bd76-51518449ca1f\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " Sep 08 15:31:45.624794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624645 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-podinfo\") pod \"b7db7c52-912e-49ed-bd76-51518449ca1f\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " Sep 08 15:31:45.624794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624681 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-data\") pod \"b7db7c52-912e-49ed-bd76-51518449ca1f\" (UID: \"b7db7c52-912e-49ed-bd76-51518449ca1f\") " Sep 08 15:31:45.624794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624755 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7db7c52-912e-49ed-bd76-51518449ca1f" (UID: "b7db7c52-912e-49ed-bd76-51518449ca1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:45.624992 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624792 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7db7c52-912e-49ed-bd76-51518449ca1f" (UID: "b7db7c52-912e-49ed-bd76-51518449ca1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:45.624992 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624843 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7db7c52-912e-49ed-bd76-51518449ca1f" (UID: "b7db7c52-912e-49ed-bd76-51518449ca1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:45.625104 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.624990 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7db7c52-912e-49ed-bd76-51518449ca1f" (UID: "b7db7c52-912e-49ed-bd76-51518449ca1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:45.625252 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.625236 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:45.625318 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.625256 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:45.625318 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.625270 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:45.625318 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.625283 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:45.625478 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.625383 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7db7c52-912e-49ed-bd76-51518449ca1f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7db7c52-912e-49ed-bd76-51518449ca1f" (UID: "b7db7c52-912e-49ed-bd76-51518449ca1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:31:45.626842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.626816 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7db7c52-912e-49ed-bd76-51518449ca1f" (UID: "b7db7c52-912e-49ed-bd76-51518449ca1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:31:45.627021 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.627002 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7db7c52-912e-49ed-bd76-51518449ca1f" (UID: "b7db7c52-912e-49ed-bd76-51518449ca1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:31:45.627270 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.627253 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7db7c52-912e-49ed-bd76-51518449ca1f" (UID: "b7db7c52-912e-49ed-bd76-51518449ca1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:31:45.627405 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.627388 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-kube-api-access-rx4h7" (OuterVolumeSpecName: "kube-api-access-rx4h7") pod "b7db7c52-912e-49ed-bd76-51518449ca1f" (UID: "b7db7c52-912e-49ed-bd76-51518449ca1f"). InnerVolumeSpecName "kube-api-access-rx4h7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:31:45.726641 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.726594 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:45.726641 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.726633 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:45.726641 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.726647 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7db7c52-912e-49ed-bd76-51518449ca1f-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:45.726874 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.726659 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx4h7\" (UniqueName: \"kubernetes.io/projected/b7db7c52-912e-49ed-bd76-51518449ca1f-kube-api-access-rx4h7\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:45.726874 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:45.726671 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7db7c52-912e-49ed-bd76-51518449ca1f-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:31:46.351643 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.351604 2578 generic.go:356] "Generic (PLEG): container finished" podID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerID="d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c" exitCode=0 Sep 08 15:31:46.351812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.351659 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" Sep 08 15:31:46.351812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.351690 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" event={"ID":"b7db7c52-912e-49ed-bd76-51518449ca1f","Type":"ContainerDied","Data":"d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c"} Sep 08 15:31:46.351812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.351748 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5" event={"ID":"b7db7c52-912e-49ed-bd76-51518449ca1f","Type":"ContainerDied","Data":"7e4ff0769ca3c2c22a7e0ae684642fbbd9119672898723bcb56b0b82bf72be6c"} Sep 08 15:31:46.351812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.351783 2578 scope.go:117] "RemoveContainer" containerID="d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c" Sep 08 15:31:46.362895 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.362879 2578 scope.go:117] "RemoveContainer" containerID="d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac" Sep 08 15:31:46.372935 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.372908 2578 scope.go:117] "RemoveContainer" containerID="d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c" Sep 08 15:31:46.373294 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:31:46.373271 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c\": container with ID starting with d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c not found: ID does not exist" containerID="d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c" Sep 08 15:31:46.373403 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.373303 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c"} err="failed to get container status \"d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c\": rpc error: code = NotFound desc = could not find container \"d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c\": container with ID starting with d0bd7c4f970c8d1033e6d0793d0bddf3ff214dc88da2619936c328b4acfc0d6c not found: ID does not exist" Sep 08 15:31:46.373403 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.373323 2578 scope.go:117] "RemoveContainer" containerID="d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac" Sep 08 15:31:46.373624 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:31:46.373605 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac\": container with ID starting with d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac not found: ID does not exist" containerID="d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac" Sep 08 15:31:46.373671 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.373632 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac"} err="failed to get container status \"d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac\": rpc error: code = NotFound desc = could not find container \"d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac\": container with ID starting with d546c5e6c6e592a6be2a04b72b80dbebc7ee0c786507a1cbe3e1bda58d1aeeac not found: ID does not exist" Sep 08 15:31:46.387227 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.387191 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5"] Sep 08 15:31:46.390758 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:46.390727 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-96c53ecc-gw-openshift-default-54b546cfb-xkxv5"] Sep 08 15:31:47.502008 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:47.501970 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" path="/var/lib/kubelet/pods/b7db7c52-912e-49ed-bd76-51518449ca1f/volumes" Sep 08 15:31:55.941189 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:55.941151 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt"] Sep 08 15:31:55.942727 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:55.942703 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" Sep 08 15:31:55.942911 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:55.942897 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" Sep 08 15:31:55.943549 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:55.943175 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" Sep 08 15:31:55.949432 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:55.949395 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" Sep 08 15:31:55.958645 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:55.958613 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:55.962788 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:55.962350 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-356350-openshift-default-dockercfg-ljmqn\"" Sep 08 15:31:55.963147 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:55.963104 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt"] Sep 08 15:31:56.038136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.035687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-socket\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.038136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.035739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-envoy\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.038136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.035802 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-podinfo\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.038136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.035852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-credential-socket\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.038136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.035892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w4zf\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-kube-api-access-7w4zf\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.038136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.035936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-data\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.038136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.035970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-certs\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.038136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.036008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-token\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.038136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.036033 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istiod-ca-cert\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.137534 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.137485 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-data\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.137835 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.137815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-certs\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.138268 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.138250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-token\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.138458 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.138439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istiod-ca-cert\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.138644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.138630 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-socket\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.138838 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.138822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-envoy\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.139271 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.138018 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-data\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.139445 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.138188 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-certs\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.139598 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.138988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-socket\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.139738 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.139284 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-podinfo\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.139932 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.139435 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istiod-ca-cert\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.140124 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.140058 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-credential-socket\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.140287 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.140269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w4zf\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-kube-api-access-7w4zf\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.144138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.140425 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-credential-socket\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.144138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.142694 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-podinfo\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.149486 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.149436 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-envoy\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.155827 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.155769 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-token\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.157316 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.157295 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w4zf\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-kube-api-access-7w4zf\") pod \"e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.272399 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.272361 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:56.580814 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.580664 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt"] Sep 08 15:31:56.585098 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:31:56.585068 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37339d43_09a2_4c9e_aaa4_2274e8fb3357.slice/crio-c0d82be45844f147541121cbc483e767414cd2a83aff300c5063a2f55e859151 WatchSource:0}: Error finding container c0d82be45844f147541121cbc483e767414cd2a83aff300c5063a2f55e859151: Status 404 returned error can't find the container with id c0d82be45844f147541121cbc483e767414cd2a83aff300c5063a2f55e859151 Sep 08 15:31:56.587367 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.587335 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:56.587467 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.587404 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:56.587467 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:56.587432 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:31:57.432147 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:57.432024 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" event={"ID":"37339d43-09a2-4c9e-aaa4-2274e8fb3357","Type":"ContainerStarted","Data":"8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec"} Sep 08 15:31:57.432147 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:57.432077 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" event={"ID":"37339d43-09a2-4c9e-aaa4-2274e8fb3357","Type":"ContainerStarted","Data":"c0d82be45844f147541121cbc483e767414cd2a83aff300c5063a2f55e859151"} Sep 08 15:31:57.493328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:57.493271 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" podStartSLOduration=2.493253388 podStartE2EDuration="2.493253388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:57.491937971 +0000 UTC m=+856.515562427" watchObservedRunningTime="2026-09-08 15:31:57.493253388 +0000 UTC m=+856.516877843" Sep 08 15:31:58.273311 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:58.273263 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:58.279001 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:58.278971 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:58.438970 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:58.438927 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:58.440430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:58.440403 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:31:59.372337 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:59.372289 2578 patch_prober.go:28] interesting pod/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 08 15:31:59.372546 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:31:59.372376 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 15:32:00.458757 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:00.458726 2578 generic.go:356] "Generic (PLEG): container finished" podID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerID="eef75bc94c5f144269d3174d129a80f38bc0f42fd58bbb491cb37d35325fcfd2" exitCode=0 Sep 08 15:32:00.459351 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:00.458796 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" event={"ID":"8f17a0c7-c290-48cb-83d3-f2d781174c58","Type":"ContainerDied","Data":"eef75bc94c5f144269d3174d129a80f38bc0f42fd58bbb491cb37d35325fcfd2"} Sep 08 15:32:00.459405 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:00.459381 2578 scope.go:117] "RemoveContainer" containerID="eef75bc94c5f144269d3174d129a80f38bc0f42fd58bbb491cb37d35325fcfd2" Sep 08 15:32:00.460577 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:00.460542 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:00.460708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:00.460624 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:00.460708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:00.460694 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:01.468602 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:01.468565 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" event={"ID":"8f17a0c7-c290-48cb-83d3-f2d781174c58","Type":"ContainerStarted","Data":"d68b217189f3306e08fff1ad72f3b1d87dd79828fea47ef320bd0c2df9d39c60"} Sep 08 15:32:01.532788 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:01.532736 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:32:01.533241 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:01.533205 2578 patch_prober.go:28] interesting pod/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Sep 08 15:32:01.533365 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:01.533268 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Sep 08 15:32:02.533064 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:02.532983 2578 patch_prober.go:28] interesting pod/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Sep 08 15:32:02.533709 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:02.533676 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Sep 08 15:32:03.533195 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:03.533146 2578 patch_prober.go:28] interesting pod/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Sep 08 15:32:03.533706 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:03.533230 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Sep 08 15:32:04.533399 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:04.533344 2578 patch_prober.go:28] interesting pod/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Sep 08 15:32:04.533902 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:04.533423 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Sep 08 15:32:05.533642 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:05.533597 2578 patch_prober.go:28] interesting pod/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Sep 08 15:32:05.534213 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:05.533675 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Sep 08 15:32:06.616849 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:06.616796 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:32:06.617333 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:06.617225 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:32:06.618510 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:06.618480 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:32:09.614633 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.614602 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s"] Sep 08 15:32:09.615056 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.615042 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" Sep 08 15:32:09.615056 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.615057 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7db7c52-912e-49ed-bd76-51518449ca1f" containerName="istio-proxy" Sep 08 15:32:09.617788 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.617766 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" Sep 08 15:32:09.621093 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.621067 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:32:09.621226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.621178 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:32:09.628625 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.627796 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s"] Sep 08 15:32:09.681621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.681575 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c83cd9-bd77-4c45-9095-e756a501cd6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s\" (UID: \"60c83cd9-bd77-4c45-9095-e756a501cd6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" Sep 08 15:32:09.782905 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.782859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c83cd9-bd77-4c45-9095-e756a501cd6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s\" (UID: \"60c83cd9-bd77-4c45-9095-e756a501cd6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" Sep 08 15:32:09.783544 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.783523 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c83cd9-bd77-4c45-9095-e756a501cd6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s\" (UID: \"60c83cd9-bd77-4c45-9095-e756a501cd6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" Sep 08 15:32:09.931823 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:09.931731 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" Sep 08 15:32:10.222834 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:10.222799 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s"] Sep 08 15:32:10.223391 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:32:10.223365 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c83cd9_bd77_4c45_9095_e756a501cd6c.slice/crio-8bf83777279c450b1b96c6a2ce3651d57ffea91acfeda2975122b2c0f341d318 WatchSource:0}: Error finding container 8bf83777279c450b1b96c6a2ce3651d57ffea91acfeda2975122b2c0f341d318: Status 404 returned error can't find the container with id 8bf83777279c450b1b96c6a2ce3651d57ffea91acfeda2975122b2c0f341d318 Sep 08 15:32:10.527401 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:10.527359 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" event={"ID":"60c83cd9-bd77-4c45-9095-e756a501cd6c","Type":"ContainerStarted","Data":"8bf83777279c450b1b96c6a2ce3651d57ffea91acfeda2975122b2c0f341d318"} Sep 08 15:32:11.535435 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:11.535398 2578 generic.go:356] "Generic (PLEG): container finished" podID="60c83cd9-bd77-4c45-9095-e756a501cd6c" containerID="eb663b3797f3aff575d25aeac77efba15f6c0c0d33f02bc0e2510e8e642f07c0" exitCode=0 Sep 08 15:32:11.535818 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:11.535473 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" event={"ID":"60c83cd9-bd77-4c45-9095-e756a501cd6c","Type":"ContainerDied","Data":"eb663b3797f3aff575d25aeac77efba15f6c0c0d33f02bc0e2510e8e642f07c0"} Sep 08 15:32:12.708608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:12.708580 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" Sep 08 15:32:12.812928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:12.812831 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c83cd9-bd77-4c45-9095-e756a501cd6c-maas-api-service-ca\") pod \"60c83cd9-bd77-4c45-9095-e756a501cd6c\" (UID: \"60c83cd9-bd77-4c45-9095-e756a501cd6c\") " Sep 08 15:32:12.813244 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:12.813221 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60c83cd9-bd77-4c45-9095-e756a501cd6c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "60c83cd9-bd77-4c45-9095-e756a501cd6c" (UID: "60c83cd9-bd77-4c45-9095-e756a501cd6c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:32:12.914356 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:12.914312 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c83cd9-bd77-4c45-9095-e756a501cd6c-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:13.556264 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:13.556229 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" Sep 08 15:32:13.556428 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:13.556265 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s" event={"ID":"60c83cd9-bd77-4c45-9095-e756a501cd6c","Type":"ContainerDied","Data":"8bf83777279c450b1b96c6a2ce3651d57ffea91acfeda2975122b2c0f341d318"} Sep 08 15:32:13.556428 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:13.556314 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bf83777279c450b1b96c6a2ce3651d57ffea91acfeda2975122b2c0f341d318" Sep 08 15:32:16.316415 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:16.316363 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s"] Sep 08 15:32:16.320434 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:16.320401 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f86ebb1f-ggf9s"] Sep 08 15:32:17.505392 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:17.505356 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60c83cd9-bd77-4c45-9095-e756a501cd6c" path="/var/lib/kubelet/pods/60c83cd9-bd77-4c45-9095-e756a501cd6c/volumes" Sep 08 15:32:17.505763 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:17.505677 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg"] Sep 08 15:32:17.505924 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:17.505899 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" containerID="cri-o://d68b217189f3306e08fff1ad72f3b1d87dd79828fea47ef320bd0c2df9d39c60" gracePeriod=30 Sep 08 15:32:18.253536 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.253497 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h"] Sep 08 15:32:18.254085 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.254067 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60c83cd9-bd77-4c45-9095-e756a501cd6c" containerName="revoke-keys" Sep 08 15:32:18.254161 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.254089 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="60c83cd9-bd77-4c45-9095-e756a501cd6c" containerName="revoke-keys" Sep 08 15:32:18.254226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.254215 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="60c83cd9-bd77-4c45-9095-e756a501cd6c" containerName="revoke-keys" Sep 08 15:32:18.258785 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.258759 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.263710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.263680 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-4f2c6181-openshift-default-dockercfg-fst77\"" Sep 08 15:32:18.271872 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.271840 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h"] Sep 08 15:32:18.386170 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.386127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-kube-api-access-wj6hb\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.386353 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.386187 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-token\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.386353 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.386223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-socket\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.386353 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.386243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-certs\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.386353 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.386259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-podinfo\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.386608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.386367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-envoy\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.386608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.386440 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istiod-ca-cert\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.386608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.386485 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-credential-socket\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.386608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.386578 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-data\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488190 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-socket\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488190 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-certs\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488462 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488211 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-podinfo\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488462 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488245 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-envoy\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488462 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istiod-ca-cert\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-credential-socket\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-data\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-kube-api-access-wj6hb\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488615 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-socket\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488844 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-token\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.488844 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488809 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-certs\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.489080 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.488970 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-data\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.489080 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.489053 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-credential-socket\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.489352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.489255 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istiod-ca-cert\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.490999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.490975 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-podinfo\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.491215 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.491197 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-envoy\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.498305 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.498273 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-kube-api-access-wj6hb\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.498418 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.498321 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-token\") pod \"e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.572551 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.572456 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:18.871918 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.871888 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h"] Sep 08 15:32:18.874101 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:32:18.874072 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb0c8a5e_823d_4aa5_9ce1_16fd448de5fa.slice/crio-6862b9e9d8e3c15b477702f15191e2c2ffb8ca9fa104317ce6fe4987e5b1b643 WatchSource:0}: Error finding container 6862b9e9d8e3c15b477702f15191e2c2ffb8ca9fa104317ce6fe4987e5b1b643: Status 404 returned error can't find the container with id 6862b9e9d8e3c15b477702f15191e2c2ffb8ca9fa104317ce6fe4987e5b1b643 Sep 08 15:32:18.876683 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.876644 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:18.876765 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.876715 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:18.876765 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:18.876745 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:19.601495 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:19.601455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" event={"ID":"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa","Type":"ContainerStarted","Data":"3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885"} Sep 08 15:32:19.601887 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:19.601522 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" event={"ID":"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa","Type":"ContainerStarted","Data":"6862b9e9d8e3c15b477702f15191e2c2ffb8ca9fa104317ce6fe4987e5b1b643"} Sep 08 15:32:19.626945 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:19.626878 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" podStartSLOduration=1.626858614 podStartE2EDuration="1.626858614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:19.625303353 +0000 UTC m=+878.648927821" watchObservedRunningTime="2026-09-08 15:32:19.626858614 +0000 UTC m=+878.650483069" Sep 08 15:32:20.573319 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.573279 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:20.578842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.578809 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:20.609735 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.609706 2578 generic.go:356] "Generic (PLEG): container finished" podID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerID="8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec" exitCode=0 Sep 08 15:32:20.610159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.609785 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" event={"ID":"37339d43-09a2-4c9e-aaa4-2274e8fb3357","Type":"ContainerDied","Data":"8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec"} Sep 08 15:32:20.610159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.610058 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:20.610428 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.610411 2578 scope.go:117] "RemoveContainer" containerID="8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec" Sep 08 15:32:20.611341 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.611196 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:20.611643 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.611618 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:20.611699 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.611686 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:20.611735 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:20.611721 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:21.618753 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:21.618716 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" event={"ID":"37339d43-09a2-4c9e-aaa4-2274e8fb3357","Type":"ContainerStarted","Data":"5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29"} Sep 08 15:32:21.619717 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:21.619692 2578 patch_prober.go:28] interesting pod/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 15:32:21.619778 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:21.619745 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 15:32:22.273086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.273047 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:32:22.273332 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.273283 2578 patch_prober.go:28] interesting pod/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Sep 08 15:32:22.273410 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.273340 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Sep 08 15:32:22.656493 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.656464 2578 generic.go:356] "Generic (PLEG): container finished" podID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerID="d68b217189f3306e08fff1ad72f3b1d87dd79828fea47ef320bd0c2df9d39c60" exitCode=0 Sep 08 15:32:22.656903 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.656542 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" event={"ID":"8f17a0c7-c290-48cb-83d3-f2d781174c58","Type":"ContainerDied","Data":"d68b217189f3306e08fff1ad72f3b1d87dd79828fea47ef320bd0c2df9d39c60"} Sep 08 15:32:22.656903 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.656622 2578 scope.go:117] "RemoveContainer" containerID="eef75bc94c5f144269d3174d129a80f38bc0f42fd58bbb491cb37d35325fcfd2" Sep 08 15:32:22.816672 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.816647 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:32:22.947138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.946491 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-data\") pod \"8f17a0c7-c290-48cb-83d3-f2d781174c58\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " Sep 08 15:32:22.947138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.946560 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-podinfo\") pod \"8f17a0c7-c290-48cb-83d3-f2d781174c58\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " Sep 08 15:32:22.947138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.946627 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-credential-socket\") pod \"8f17a0c7-c290-48cb-83d3-f2d781174c58\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " Sep 08 15:32:22.947138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.946666 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db4jb\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-kube-api-access-db4jb\") pod \"8f17a0c7-c290-48cb-83d3-f2d781174c58\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " Sep 08 15:32:22.947138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.946695 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-token\") pod \"8f17a0c7-c290-48cb-83d3-f2d781174c58\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " Sep 08 15:32:22.947138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.946785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-envoy\") pod \"8f17a0c7-c290-48cb-83d3-f2d781174c58\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " Sep 08 15:32:22.947138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.946816 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f17a0c7-c290-48cb-83d3-f2d781174c58-istiod-ca-cert\") pod \"8f17a0c7-c290-48cb-83d3-f2d781174c58\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " Sep 08 15:32:22.947138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.946840 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-socket\") pod \"8f17a0c7-c290-48cb-83d3-f2d781174c58\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " Sep 08 15:32:22.947138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.946865 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-certs\") pod \"8f17a0c7-c290-48cb-83d3-f2d781174c58\" (UID: \"8f17a0c7-c290-48cb-83d3-f2d781174c58\") " Sep 08 15:32:22.947771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.947441 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f17a0c7-c290-48cb-83d3-f2d781174c58" (UID: "8f17a0c7-c290-48cb-83d3-f2d781174c58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:22.947771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.947605 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f17a0c7-c290-48cb-83d3-f2d781174c58" (UID: "8f17a0c7-c290-48cb-83d3-f2d781174c58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:22.951141 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.948423 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f17a0c7-c290-48cb-83d3-f2d781174c58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f17a0c7-c290-48cb-83d3-f2d781174c58" (UID: "8f17a0c7-c290-48cb-83d3-f2d781174c58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:32:22.951141 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.948803 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f17a0c7-c290-48cb-83d3-f2d781174c58" (UID: "8f17a0c7-c290-48cb-83d3-f2d781174c58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:22.951141 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.948946 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f17a0c7-c290-48cb-83d3-f2d781174c58" (UID: "8f17a0c7-c290-48cb-83d3-f2d781174c58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:22.952428 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.952007 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f17a0c7-c290-48cb-83d3-f2d781174c58" (UID: "8f17a0c7-c290-48cb-83d3-f2d781174c58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:32:22.952428 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.952357 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f17a0c7-c290-48cb-83d3-f2d781174c58" (UID: "8f17a0c7-c290-48cb-83d3-f2d781174c58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:32:22.953526 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.953491 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-kube-api-access-db4jb" (OuterVolumeSpecName: "kube-api-access-db4jb") pod "8f17a0c7-c290-48cb-83d3-f2d781174c58" (UID: "8f17a0c7-c290-48cb-83d3-f2d781174c58"). InnerVolumeSpecName "kube-api-access-db4jb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:32:22.953933 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:22.953915 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f17a0c7-c290-48cb-83d3-f2d781174c58" (UID: "8f17a0c7-c290-48cb-83d3-f2d781174c58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:23.048947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.048728 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:23.048947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.048767 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:23.048947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.048781 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:23.048947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.048797 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db4jb\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-kube-api-access-db4jb\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:23.048947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.048810 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:23.048947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.048824 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:23.048947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.048837 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f17a0c7-c290-48cb-83d3-f2d781174c58-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:23.048947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.048848 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:23.048947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.048860 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f17a0c7-c290-48cb-83d3-f2d781174c58-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:23.273871 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.273830 2578 patch_prober.go:28] interesting pod/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Sep 08 15:32:23.274056 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.273893 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Sep 08 15:32:23.665492 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.665452 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" event={"ID":"8f17a0c7-c290-48cb-83d3-f2d781174c58","Type":"ContainerDied","Data":"5e0dec7c38cb8fd544f22ecee64d522a209720e2343c17a1b008db5c034d341f"} Sep 08 15:32:23.665956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.665537 2578 scope.go:117] "RemoveContainer" containerID="d68b217189f3306e08fff1ad72f3b1d87dd79828fea47ef320bd0c2df9d39c60" Sep 08 15:32:23.665956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.665460 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg" Sep 08 15:32:23.698312 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.698273 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg"] Sep 08 15:32:23.706526 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:23.706492 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f86ebb1f-openshift-default-5c85dd5cb4-nsvkg"] Sep 08 15:32:24.273913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:24.273863 2578 patch_prober.go:28] interesting pod/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Sep 08 15:32:24.274097 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:24.273957 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Sep 08 15:32:25.273879 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:25.273842 2578 patch_prober.go:28] interesting pod/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Sep 08 15:32:25.274309 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:25.273899 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Sep 08 15:32:25.504042 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:25.504004 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" path="/var/lib/kubelet/pods/8f17a0c7-c290-48cb-83d3-f2d781174c58/volumes" Sep 08 15:32:26.273539 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:26.273495 2578 patch_prober.go:28] interesting pod/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Sep 08 15:32:26.273833 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:26.273559 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Sep 08 15:32:27.281047 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:27.281010 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:32:27.281513 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:27.281463 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:32:27.283091 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:27.283065 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:32:40.790012 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:40.789925 2578 generic.go:356] "Generic (PLEG): container finished" podID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerID="3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885" exitCode=0 Sep 08 15:32:40.790526 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:40.790003 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" event={"ID":"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa","Type":"ContainerDied","Data":"3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885"} Sep 08 15:32:40.790638 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:40.790619 2578 scope.go:117] "RemoveContainer" containerID="3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885" Sep 08 15:32:40.791812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:40.791781 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:40.791883 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:40.791864 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:40.791928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:40.791899 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:32:41.798370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:41.798280 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" event={"ID":"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa","Type":"ContainerStarted","Data":"813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111"} Sep 08 15:32:42.572906 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:42.572871 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:42.573133 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:42.573080 2578 patch_prober.go:28] interesting pod/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 08 15:32:42.573234 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:42.573177 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 08 15:32:43.572817 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:43.572778 2578 patch_prober.go:28] interesting pod/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 08 15:32:43.573223 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:43.572846 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 08 15:32:44.572853 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:44.572808 2578 patch_prober.go:28] interesting pod/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 08 15:32:44.573342 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:44.572873 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 08 15:32:45.573291 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.573245 2578 patch_prober.go:28] interesting pod/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 08 15:32:45.573705 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.573316 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 08 15:32:45.875389 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.875286 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc"] Sep 08 15:32:45.875841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.875823 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" Sep 08 15:32:45.875931 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.875844 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" Sep 08 15:32:45.875931 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.875868 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" Sep 08 15:32:45.875931 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.875877 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" Sep 08 15:32:45.876089 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.875973 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" Sep 08 15:32:45.876089 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.875988 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f17a0c7-c290-48cb-83d3-f2d781174c58" containerName="istio-proxy" Sep 08 15:32:45.880562 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.880539 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" Sep 08 15:32:45.884978 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.884953 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:32:45.885343 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.885323 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:32:45.887505 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.887483 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc"] Sep 08 15:32:45.991509 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:45.991470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0a4cf90-5073-4801-946b-70552295b30a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc\" (UID: \"b0a4cf90-5073-4801-946b-70552295b30a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" Sep 08 15:32:46.092730 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:46.092686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0a4cf90-5073-4801-946b-70552295b30a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc\" (UID: \"b0a4cf90-5073-4801-946b-70552295b30a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" Sep 08 15:32:46.093376 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:46.093354 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0a4cf90-5073-4801-946b-70552295b30a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc\" (UID: \"b0a4cf90-5073-4801-946b-70552295b30a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" Sep 08 15:32:46.193180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:46.193052 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" Sep 08 15:32:46.485283 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:46.485249 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc"] Sep 08 15:32:46.487909 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:32:46.487880 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0a4cf90_5073_4801_946b_70552295b30a.slice/crio-51a1da1c91cb30d92a1829c442198c85cb96b97af309f26be917a1976cdb0262 WatchSource:0}: Error finding container 51a1da1c91cb30d92a1829c442198c85cb96b97af309f26be917a1976cdb0262: Status 404 returned error can't find the container with id 51a1da1c91cb30d92a1829c442198c85cb96b97af309f26be917a1976cdb0262 Sep 08 15:32:46.573403 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:46.573358 2578 patch_prober.go:28] interesting pod/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 08 15:32:46.573795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:46.573430 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 08 15:32:46.836176 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:46.836130 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" event={"ID":"b0a4cf90-5073-4801-946b-70552295b30a","Type":"ContainerStarted","Data":"51a1da1c91cb30d92a1829c442198c85cb96b97af309f26be917a1976cdb0262"} Sep 08 15:32:47.577590 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:47.577554 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:47.578031 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:47.577837 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:47.578701 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:47.578684 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:47.843466 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:47.843434 2578 generic.go:356] "Generic (PLEG): container finished" podID="b0a4cf90-5073-4801-946b-70552295b30a" containerID="9d2b8eb080ba10a9166d825ec19f3cc2a1ebc79b9fc089470bd7ef2adf1dc9b2" exitCode=0 Sep 08 15:32:47.843668 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:47.843516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" event={"ID":"b0a4cf90-5073-4801-946b-70552295b30a","Type":"ContainerDied","Data":"9d2b8eb080ba10a9166d825ec19f3cc2a1ebc79b9fc089470bd7ef2adf1dc9b2"} Sep 08 15:32:48.991675 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:48.991645 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" Sep 08 15:32:49.123874 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:49.123780 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0a4cf90-5073-4801-946b-70552295b30a-maas-api-service-ca\") pod \"b0a4cf90-5073-4801-946b-70552295b30a\" (UID: \"b0a4cf90-5073-4801-946b-70552295b30a\") " Sep 08 15:32:49.124190 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:49.124167 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0a4cf90-5073-4801-946b-70552295b30a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b0a4cf90-5073-4801-946b-70552295b30a" (UID: "b0a4cf90-5073-4801-946b-70552295b30a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:32:49.225600 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:49.225554 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0a4cf90-5073-4801-946b-70552295b30a-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:49.858236 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:49.858155 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" Sep 08 15:32:49.858236 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:49.858183 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc" event={"ID":"b0a4cf90-5073-4801-946b-70552295b30a","Type":"ContainerDied","Data":"51a1da1c91cb30d92a1829c442198c85cb96b97af309f26be917a1976cdb0262"} Sep 08 15:32:49.858236 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:49.858233 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51a1da1c91cb30d92a1829c442198c85cb96b97af309f26be917a1976cdb0262" Sep 08 15:32:53.175593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:53.175558 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc"] Sep 08 15:32:53.178175 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:53.178151 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4f2c6181-vvwlc"] Sep 08 15:32:53.325266 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:53.325225 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h"] Sep 08 15:32:53.325510 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:53.325479 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" containerID="cri-o://813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111" gracePeriod=30 Sep 08 15:32:53.502961 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:53.502924 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0a4cf90-5073-4801-946b-70552295b30a" path="/var/lib/kubelet/pods/b0a4cf90-5073-4801-946b-70552295b30a/volumes" Sep 08 15:32:58.584812 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.584783 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:58.725649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725608 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-kube-api-access-wj6hb\") pod \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " Sep 08 15:32:58.725649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725653 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-podinfo\") pod \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " Sep 08 15:32:58.725922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725674 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-credential-socket\") pod \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " Sep 08 15:32:58.725922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725696 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-certs\") pod \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " Sep 08 15:32:58.725922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725764 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istiod-ca-cert\") pod \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " Sep 08 15:32:58.725922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725781 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-token\") pod \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " Sep 08 15:32:58.725922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725852 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-data\") pod \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " Sep 08 15:32:58.725922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725910 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-envoy\") pod \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " Sep 08 15:32:58.726275 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725953 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-socket\") pod \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\" (UID: \"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa\") " Sep 08 15:32:58.726275 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.725954 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" (UID: "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:58.726275 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.726190 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" (UID: "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:58.726433 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.726320 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" (UID: "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:32:58.726648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.726432 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-data" (OuterVolumeSpecName: "istio-data") pod "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" (UID: "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:58.726648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.726396 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" (UID: "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:58.726648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.726327 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:58.726648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.726475 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:58.728253 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.728212 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" (UID: "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:32:58.728386 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.728322 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" (UID: "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:32:58.728621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.728594 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-kube-api-access-wj6hb" (OuterVolumeSpecName: "kube-api-access-wj6hb") pod "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" (UID: "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa"). InnerVolumeSpecName "kube-api-access-wj6hb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:32:58.728621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.728610 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" (UID: "db0c8a5e-823d-4aa5-9ce1-16fd448de5fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:32:58.827544 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.827501 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:58.827544 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.827535 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:58.827544 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.827543 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:58.827544 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.827553 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-kube-api-access-wj6hb\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:58.827824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.827564 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:58.827824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.827572 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:58.827824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.827579 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:32:58.916390 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.916358 2578 generic.go:356] "Generic (PLEG): container finished" podID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerID="813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111" exitCode=0 Sep 08 15:32:58.916573 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.916418 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" Sep 08 15:32:58.916573 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.916446 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" event={"ID":"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa","Type":"ContainerDied","Data":"813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111"} Sep 08 15:32:58.916573 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.916529 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h" event={"ID":"db0c8a5e-823d-4aa5-9ce1-16fd448de5fa","Type":"ContainerDied","Data":"6862b9e9d8e3c15b477702f15191e2c2ffb8ca9fa104317ce6fe4987e5b1b643"} Sep 08 15:32:58.916573 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.916554 2578 scope.go:117] "RemoveContainer" containerID="813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111" Sep 08 15:32:58.926617 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.926596 2578 scope.go:117] "RemoveContainer" containerID="3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885" Sep 08 15:32:58.935680 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.935657 2578 scope.go:117] "RemoveContainer" containerID="813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111" Sep 08 15:32:58.935969 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:32:58.935949 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111\": container with ID starting with 813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111 not found: ID does not exist" containerID="813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111" Sep 08 15:32:58.936068 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.935978 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111"} err="failed to get container status \"813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111\": rpc error: code = NotFound desc = could not find container \"813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111\": container with ID starting with 813c7ae5507c97142eeade2c8d32d82555e188955888a03a2d38890601d83111 not found: ID does not exist" Sep 08 15:32:58.936068 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.935997 2578 scope.go:117] "RemoveContainer" containerID="3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885" Sep 08 15:32:58.936320 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:32:58.936303 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885\": container with ID starting with 3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885 not found: ID does not exist" containerID="3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885" Sep 08 15:32:58.936375 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.936329 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885"} err="failed to get container status \"3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885\": rpc error: code = NotFound desc = could not find container \"3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885\": container with ID starting with 3fbb367395e6f7d2a1b3d515df3e47fcf529198a54374d902b590c8f86782885 not found: ID does not exist" Sep 08 15:32:58.941896 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.941873 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h"] Sep 08 15:32:58.952399 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:58.952368 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-4f2c6181-openshift-default-857b89c7d8-56g9h"] Sep 08 15:32:59.502019 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:32:59.501985 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" path="/var/lib/kubelet/pods/db0c8a5e-823d-4aa5-9ce1-16fd448de5fa/volumes" Sep 08 15:33:32.122886 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.122847 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz"] Sep 08 15:33:32.123420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.123327 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" Sep 08 15:33:32.123420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.123340 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" Sep 08 15:33:32.123420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.123360 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a4cf90-5073-4801-946b-70552295b30a" containerName="revoke-keys" Sep 08 15:33:32.123420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.123367 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a4cf90-5073-4801-946b-70552295b30a" containerName="revoke-keys" Sep 08 15:33:32.123420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.123381 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" Sep 08 15:33:32.123420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.123388 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" Sep 08 15:33:32.123698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.123479 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" Sep 08 15:33:32.123698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.123489 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0a4cf90-5073-4801-946b-70552295b30a" containerName="revoke-keys" Sep 08 15:33:32.123698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.123497 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0c8a5e-823d-4aa5-9ce1-16fd448de5fa" containerName="istio-proxy" Sep 08 15:33:32.126420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.126398 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" Sep 08 15:33:32.129150 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.129129 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:33:32.129286 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.129148 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:33:32.136345 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.136322 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz"] Sep 08 15:33:32.257874 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.257831 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378f547e-3ba2-4c8d-8d6e-484c11134bdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-356350-29xzz\" (UID: \"378f547e-3ba2-4c8d-8d6e-484c11134bdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" Sep 08 15:33:32.359251 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.359209 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378f547e-3ba2-4c8d-8d6e-484c11134bdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-356350-29xzz\" (UID: \"378f547e-3ba2-4c8d-8d6e-484c11134bdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" Sep 08 15:33:32.359837 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.359813 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378f547e-3ba2-4c8d-8d6e-484c11134bdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-356350-29xzz\" (UID: \"378f547e-3ba2-4c8d-8d6e-484c11134bdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" Sep 08 15:33:32.438296 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.438219 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" Sep 08 15:33:32.723461 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:32.723429 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz"] Sep 08 15:33:32.725078 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:33:32.725054 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod378f547e_3ba2_4c8d_8d6e_484c11134bdb.slice/crio-cedfd2543afcfeb7ca33f8dba1a3ccab38363f2819e72b13ab644c707ea5601d WatchSource:0}: Error finding container cedfd2543afcfeb7ca33f8dba1a3ccab38363f2819e72b13ab644c707ea5601d: Status 404 returned error can't find the container with id cedfd2543afcfeb7ca33f8dba1a3ccab38363f2819e72b13ab644c707ea5601d Sep 08 15:33:33.135879 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:33.135841 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" event={"ID":"378f547e-3ba2-4c8d-8d6e-484c11134bdb","Type":"ContainerStarted","Data":"0e46c9bed1a19e8526d59244afca36f1c3da5811d89a390aa91adf4cebacc491"} Sep 08 15:33:33.136274 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:33.135906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" event={"ID":"378f547e-3ba2-4c8d-8d6e-484c11134bdb","Type":"ContainerStarted","Data":"cedfd2543afcfeb7ca33f8dba1a3ccab38363f2819e72b13ab644c707ea5601d"} Sep 08 15:33:33.169269 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:33.169204 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" podStartSLOduration=1.169186259 podStartE2EDuration="1.169186259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:33.166981649 +0000 UTC m=+952.190606105" watchObservedRunningTime="2026-09-08 15:33:33.169186259 +0000 UTC m=+952.192810715" Sep 08 15:33:34.143673 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:34.143640 2578 generic.go:356] "Generic (PLEG): container finished" podID="378f547e-3ba2-4c8d-8d6e-484c11134bdb" containerID="0e46c9bed1a19e8526d59244afca36f1c3da5811d89a390aa91adf4cebacc491" exitCode=0 Sep 08 15:33:34.144064 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:34.143714 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" event={"ID":"378f547e-3ba2-4c8d-8d6e-484c11134bdb","Type":"ContainerDied","Data":"0e46c9bed1a19e8526d59244afca36f1c3da5811d89a390aa91adf4cebacc491"} Sep 08 15:33:35.289182 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:35.289152 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" Sep 08 15:33:35.390764 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:35.390728 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378f547e-3ba2-4c8d-8d6e-484c11134bdb-maas-api-service-ca\") pod \"378f547e-3ba2-4c8d-8d6e-484c11134bdb\" (UID: \"378f547e-3ba2-4c8d-8d6e-484c11134bdb\") " Sep 08 15:33:35.391072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:35.391048 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/378f547e-3ba2-4c8d-8d6e-484c11134bdb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "378f547e-3ba2-4c8d-8d6e-484c11134bdb" (UID: "378f547e-3ba2-4c8d-8d6e-484c11134bdb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:33:35.391184 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:35.391160 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378f547e-3ba2-4c8d-8d6e-484c11134bdb-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:36.157153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:36.157098 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" event={"ID":"378f547e-3ba2-4c8d-8d6e-484c11134bdb","Type":"ContainerDied","Data":"cedfd2543afcfeb7ca33f8dba1a3ccab38363f2819e72b13ab644c707ea5601d"} Sep 08 15:33:36.157355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:36.157155 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz" Sep 08 15:33:36.157355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:36.157163 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cedfd2543afcfeb7ca33f8dba1a3ccab38363f2819e72b13ab644c707ea5601d" Sep 08 15:33:43.000822 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:43.000739 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz"] Sep 08 15:33:43.003838 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:43.003803 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-356350-29xzz"] Sep 08 15:33:43.502639 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:43.502605 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="378f547e-3ba2-4c8d-8d6e-484c11134bdb" path="/var/lib/kubelet/pods/378f547e-3ba2-4c8d-8d6e-484c11134bdb/volumes" Sep 08 15:33:49.708419 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:49.708379 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt"] Sep 08 15:33:49.708857 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:49.708641 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" containerID="cri-o://5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29" gracePeriod=30 Sep 08 15:33:50.553606 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.553567 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4"] Sep 08 15:33:50.554013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.554001 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="378f547e-3ba2-4c8d-8d6e-484c11134bdb" containerName="revoke-keys" Sep 08 15:33:50.554067 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.554015 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="378f547e-3ba2-4c8d-8d6e-484c11134bdb" containerName="revoke-keys" Sep 08 15:33:50.554103 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.554097 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="378f547e-3ba2-4c8d-8d6e-484c11134bdb" containerName="revoke-keys" Sep 08 15:33:50.558711 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.558676 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.562144 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.562097 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-86d73c-openshift-default-dockercfg-kt9jn\"" Sep 08 15:33:50.571415 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.571303 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4"] Sep 08 15:33:50.743870 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.743826 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-socket\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.743870 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.743868 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-token\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.744326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.743962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk6rv\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-kube-api-access-pk6rv\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.744326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.744022 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-podinfo\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.744326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.744069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-credential-socket\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.744326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.744089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istiod-ca-cert\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.744326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.744205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-envoy\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.744326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.744241 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-certs\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.744326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.744261 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-data\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.845793 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.845696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-socket\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.845793 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.845739 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-token\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.845986 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.845876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk6rv\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-kube-api-access-pk6rv\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.845986 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.845964 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-podinfo\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846083 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-credential-socket\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846083 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istiod-ca-cert\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846310 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846087 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-socket\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846310 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-envoy\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846310 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846224 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-certs\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846310 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846248 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-data\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846526 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846405 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-credential-socket\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846586 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846567 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-certs\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846640 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846601 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-data\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.846802 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.846738 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istiod-ca-cert\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.848504 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.848481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-envoy\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.848746 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.848728 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-podinfo\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.859614 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.859580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-token\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.859777 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.859755 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk6rv\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-kube-api-access-pk6rv\") pod \"e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:50.871757 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:50.871734 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:51.158742 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:51.158707 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4"] Sep 08 15:33:51.159955 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:33:51.159930 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f13c08_1548_4fb3_a8b3_6f384746ada3.slice/crio-d472f1454b9bf0897f52526bcbbd6378b77cb6c748e579c2cf0c7c9d1cf27846 WatchSource:0}: Error finding container d472f1454b9bf0897f52526bcbbd6378b77cb6c748e579c2cf0c7c9d1cf27846: Status 404 returned error can't find the container with id d472f1454b9bf0897f52526bcbbd6378b77cb6c748e579c2cf0c7c9d1cf27846 Sep 08 15:33:51.162198 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:51.162165 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:33:51.162300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:51.162247 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:33:51.162300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:51.162292 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:33:51.260787 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:51.260743 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" event={"ID":"f9f13c08-1548-4fb3-a8b3-6f384746ada3","Type":"ContainerStarted","Data":"22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150"} Sep 08 15:33:51.260958 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:51.260811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" event={"ID":"f9f13c08-1548-4fb3-a8b3-6f384746ada3","Type":"ContainerStarted","Data":"d472f1454b9bf0897f52526bcbbd6378b77cb6c748e579c2cf0c7c9d1cf27846"} Sep 08 15:33:51.287604 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:51.287545 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" podStartSLOduration=1.28752698 podStartE2EDuration="1.28752698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:51.284882266 +0000 UTC m=+970.308506722" watchObservedRunningTime="2026-09-08 15:33:51.28752698 +0000 UTC m=+970.311151436" Sep 08 15:33:51.872103 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:51.872059 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:52.879414 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:52.879384 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:53.272384 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:53.272350 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:53.273398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:53.273380 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:33:54.958285 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.958255 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:33:54.980345 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980264 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-data\") pod \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " Sep 08 15:33:54.980345 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980309 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w4zf\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-kube-api-access-7w4zf\") pod \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " Sep 08 15:33:54.980345 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980332 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-certs\") pod \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " Sep 08 15:33:54.980567 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980433 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-podinfo\") pod \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " Sep 08 15:33:54.980567 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980464 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-credential-socket\") pod \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " Sep 08 15:33:54.980567 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980495 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-token\") pod \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " Sep 08 15:33:54.980725 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980569 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-data" (OuterVolumeSpecName: "istio-data") pod "37339d43-09a2-4c9e-aaa4-2274e8fb3357" (UID: "37339d43-09a2-4c9e-aaa4-2274e8fb3357"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:33:54.980725 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980585 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-socket\") pod \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " Sep 08 15:33:54.980725 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980626 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-envoy\") pod \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " Sep 08 15:33:54.980725 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980660 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istiod-ca-cert\") pod \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\" (UID: \"37339d43-09a2-4c9e-aaa4-2274e8fb3357\") " Sep 08 15:33:54.980933 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980738 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37339d43-09a2-4c9e-aaa4-2274e8fb3357" (UID: "37339d43-09a2-4c9e-aaa4-2274e8fb3357"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:33:54.980933 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980756 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37339d43-09a2-4c9e-aaa4-2274e8fb3357" (UID: "37339d43-09a2-4c9e-aaa4-2274e8fb3357"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:33:54.980933 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.980849 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37339d43-09a2-4c9e-aaa4-2274e8fb3357" (UID: "37339d43-09a2-4c9e-aaa4-2274e8fb3357"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:33:54.981137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.981068 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:54.981137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.981090 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:54.981137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.981105 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:54.981261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.981152 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:54.981261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.981148 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37339d43-09a2-4c9e-aaa4-2274e8fb3357" (UID: "37339d43-09a2-4c9e-aaa4-2274e8fb3357"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:33:54.982786 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.982760 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37339d43-09a2-4c9e-aaa4-2274e8fb3357" (UID: "37339d43-09a2-4c9e-aaa4-2274e8fb3357"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:33:54.982894 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.982872 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-token" (OuterVolumeSpecName: "istio-token") pod "37339d43-09a2-4c9e-aaa4-2274e8fb3357" (UID: "37339d43-09a2-4c9e-aaa4-2274e8fb3357"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:33:54.983329 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.983306 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37339d43-09a2-4c9e-aaa4-2274e8fb3357" (UID: "37339d43-09a2-4c9e-aaa4-2274e8fb3357"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:33:54.983329 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:54.983323 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-kube-api-access-7w4zf" (OuterVolumeSpecName: "kube-api-access-7w4zf") pod "37339d43-09a2-4c9e-aaa4-2274e8fb3357" (UID: "37339d43-09a2-4c9e-aaa4-2274e8fb3357"). InnerVolumeSpecName "kube-api-access-7w4zf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:33:55.082013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.081977 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:55.082013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.082008 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:55.082013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.082019 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:55.082272 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.082027 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37339d43-09a2-4c9e-aaa4-2274e8fb3357-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:55.082272 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.082035 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w4zf\" (UniqueName: \"kubernetes.io/projected/37339d43-09a2-4c9e-aaa4-2274e8fb3357-kube-api-access-7w4zf\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:33:55.286271 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.286187 2578 generic.go:356] "Generic (PLEG): container finished" podID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerID="5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29" exitCode=0 Sep 08 15:33:55.286271 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.286235 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" Sep 08 15:33:55.286271 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.286263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" event={"ID":"37339d43-09a2-4c9e-aaa4-2274e8fb3357","Type":"ContainerDied","Data":"5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29"} Sep 08 15:33:55.286485 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.286314 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt" event={"ID":"37339d43-09a2-4c9e-aaa4-2274e8fb3357","Type":"ContainerDied","Data":"c0d82be45844f147541121cbc483e767414cd2a83aff300c5063a2f55e859151"} Sep 08 15:33:55.286485 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.286339 2578 scope.go:117] "RemoveContainer" containerID="5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29" Sep 08 15:33:55.300862 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.300841 2578 scope.go:117] "RemoveContainer" containerID="8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec" Sep 08 15:33:55.309499 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.309482 2578 scope.go:117] "RemoveContainer" containerID="5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29" Sep 08 15:33:55.309810 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:33:55.309792 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29\": container with ID starting with 5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29 not found: ID does not exist" containerID="5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29" Sep 08 15:33:55.309858 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.309820 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29"} err="failed to get container status \"5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29\": rpc error: code = NotFound desc = could not find container \"5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29\": container with ID starting with 5768b9da1660dd8ffa92f340fdd951da7e81331afa77a11613fc6b658a208a29 not found: ID does not exist" Sep 08 15:33:55.309858 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.309837 2578 scope.go:117] "RemoveContainer" containerID="8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec" Sep 08 15:33:55.310074 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:33:55.310058 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec\": container with ID starting with 8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec not found: ID does not exist" containerID="8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec" Sep 08 15:33:55.310216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.310077 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec"} err="failed to get container status \"8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec\": rpc error: code = NotFound desc = could not find container \"8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec\": container with ID starting with 8fc4e4e9cbe4c59f521c8ca492f5387c6cd23c20d763e5d89eb929f7edb021ec not found: ID does not exist" Sep 08 15:33:55.313960 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.313935 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt"] Sep 08 15:33:55.318540 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.318507 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-356350-openshift-default-85f8787bb8-x4jmt"] Sep 08 15:33:55.502233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:33:55.502194 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" path="/var/lib/kubelet/pods/37339d43-09a2-4c9e-aaa4-2274e8fb3357/volumes" Sep 08 15:34:26.249375 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:26.249338 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4"] Sep 08 15:34:26.249934 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:26.249598 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" podUID="f9f13c08-1548-4fb3-a8b3-6f384746ada3" containerName="istio-proxy" containerID="cri-o://22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150" gracePeriod=30 Sep 08 15:34:31.503871 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.503840 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:34:31.524755 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.524655 2578 generic.go:356] "Generic (PLEG): container finished" podID="f9f13c08-1548-4fb3-a8b3-6f384746ada3" containerID="22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150" exitCode=0 Sep 08 15:34:31.525032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.524694 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" event={"ID":"f9f13c08-1548-4fb3-a8b3-6f384746ada3","Type":"ContainerDied","Data":"22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150"} Sep 08 15:34:31.525205 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.525079 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" event={"ID":"f9f13c08-1548-4fb3-a8b3-6f384746ada3","Type":"ContainerDied","Data":"d472f1454b9bf0897f52526bcbbd6378b77cb6c748e579c2cf0c7c9d1cf27846"} Sep 08 15:34:31.525205 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.524725 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4" Sep 08 15:34:31.525205 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.525142 2578 scope.go:117] "RemoveContainer" containerID="22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150" Sep 08 15:34:31.537029 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.536997 2578 scope.go:117] "RemoveContainer" containerID="22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150" Sep 08 15:34:31.537424 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:34:31.537396 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150\": container with ID starting with 22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150 not found: ID does not exist" containerID="22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150" Sep 08 15:34:31.537542 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.537437 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150"} err="failed to get container status \"22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150\": rpc error: code = NotFound desc = could not find container \"22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150\": container with ID starting with 22b943917fe2a50843d2107e637480a113b25f1fc414bf9d484460aef7f71150 not found: ID does not exist" Sep 08 15:34:31.547286 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547256 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-data\") pod \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " Sep 08 15:34:31.547405 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547360 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-certs\") pod \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " Sep 08 15:34:31.547451 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547414 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-token\") pod \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " Sep 08 15:34:31.547451 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547441 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk6rv\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-kube-api-access-pk6rv\") pod \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " Sep 08 15:34:31.547556 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547481 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istiod-ca-cert\") pod \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " Sep 08 15:34:31.547556 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547517 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-credential-socket\") pod \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " Sep 08 15:34:31.547556 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547534 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9f13c08-1548-4fb3-a8b3-6f384746ada3" (UID: "f9f13c08-1548-4fb3-a8b3-6f384746ada3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:34:31.547708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547558 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-socket\") pod \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " Sep 08 15:34:31.547708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547589 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-podinfo\") pod \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " Sep 08 15:34:31.547708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547617 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9f13c08-1548-4fb3-a8b3-6f384746ada3" (UID: "f9f13c08-1548-4fb3-a8b3-6f384746ada3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:34:31.547708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547634 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-envoy\") pod \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\" (UID: \"f9f13c08-1548-4fb3-a8b3-6f384746ada3\") " Sep 08 15:34:31.548153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.547988 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:31.548153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.548026 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:31.548153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.548030 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9f13c08-1548-4fb3-a8b3-6f384746ada3" (UID: "f9f13c08-1548-4fb3-a8b3-6f384746ada3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:34:31.548153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.548098 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9f13c08-1548-4fb3-a8b3-6f384746ada3" (UID: "f9f13c08-1548-4fb3-a8b3-6f384746ada3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:34:31.548637 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.548247 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9f13c08-1548-4fb3-a8b3-6f384746ada3" (UID: "f9f13c08-1548-4fb3-a8b3-6f384746ada3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:34:31.550044 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.550023 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9f13c08-1548-4fb3-a8b3-6f384746ada3" (UID: "f9f13c08-1548-4fb3-a8b3-6f384746ada3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:34:31.550196 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.550087 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9f13c08-1548-4fb3-a8b3-6f384746ada3" (UID: "f9f13c08-1548-4fb3-a8b3-6f384746ada3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:34:31.550410 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.550385 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-kube-api-access-pk6rv" (OuterVolumeSpecName: "kube-api-access-pk6rv") pod "f9f13c08-1548-4fb3-a8b3-6f384746ada3" (UID: "f9f13c08-1548-4fb3-a8b3-6f384746ada3"). InnerVolumeSpecName "kube-api-access-pk6rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:34:31.550531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.550507 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9f13c08-1548-4fb3-a8b3-6f384746ada3" (UID: "f9f13c08-1548-4fb3-a8b3-6f384746ada3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:34:31.649608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.649570 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:31.649608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.649603 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk6rv\" (UniqueName: \"kubernetes.io/projected/f9f13c08-1548-4fb3-a8b3-6f384746ada3-kube-api-access-pk6rv\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:31.649608 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.649614 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:31.649921 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.649624 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:31.649921 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.649638 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:31.649921 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.649648 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:31.649921 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.649656 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f13c08-1548-4fb3-a8b3-6f384746ada3-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:31.856164 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.856127 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4"] Sep 08 15:34:31.860702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:31.860670 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-86d73c-openshift-default-7996c4b59d-s64z4"] Sep 08 15:34:32.105585 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.105495 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4"] Sep 08 15:34:32.105962 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.105950 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" Sep 08 15:34:32.106005 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.105964 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" Sep 08 15:34:32.106005 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.105978 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9f13c08-1548-4fb3-a8b3-6f384746ada3" containerName="istio-proxy" Sep 08 15:34:32.106005 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.105983 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9f13c08-1548-4fb3-a8b3-6f384746ada3" containerName="istio-proxy" Sep 08 15:34:32.106005 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.105991 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" Sep 08 15:34:32.106005 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.105997 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" Sep 08 15:34:32.106180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.106058 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" Sep 08 15:34:32.106180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.106067 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="37339d43-09a2-4c9e-aaa4-2274e8fb3357" containerName="istio-proxy" Sep 08 15:34:32.106180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.106073 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9f13c08-1548-4fb3-a8b3-6f384746ada3" containerName="istio-proxy" Sep 08 15:34:32.110408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.110391 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.113396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.113375 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9fbfa439-openshift-default-dockercfg-dl7rs\"" Sep 08 15:34:32.129692 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.129660 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4"] Sep 08 15:34:32.153130 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.153083 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-certs\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.153294 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.153206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-token\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.153294 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.153262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-credential-socket\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.153294 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.153280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-envoy\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.153401 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.153304 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqxvt\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-kube-api-access-lqxvt\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.153401 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.153348 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istiod-ca-cert\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.153401 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.153367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-data\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.153500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.153402 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-socket\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.153500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.153429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-podinfo\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.254428 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-certs\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.254634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-token\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.254634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-credential-socket\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.254634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-envoy\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.254634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254531 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqxvt\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-kube-api-access-lqxvt\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.254634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istiod-ca-cert\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.254634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-data\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.254634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254636 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-socket\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.255000 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-podinfo\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.255000 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.254943 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-certs\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.255171 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.255149 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-credential-socket\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.255236 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.255185 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-socket\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.255274 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.255256 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-data\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.255379 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.255350 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istiod-ca-cert\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.257041 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.257017 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-envoy\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.257331 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.257315 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-podinfo\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.266362 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.266331 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-token\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.266460 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.266399 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqxvt\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-kube-api-access-lqxvt\") pod \"e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.421797 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.421694 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:32.704369 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.704335 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4"] Sep 08 15:34:32.705009 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:34:32.704978 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode163be32_1a6a_4e87_b09b_a054d7f4e3a9.slice/crio-4a439e8bee88fd64e428629fe629c19b3415f2a2e779271405efba4bdff54861 WatchSource:0}: Error finding container 4a439e8bee88fd64e428629fe629c19b3415f2a2e779271405efba4bdff54861: Status 404 returned error can't find the container with id 4a439e8bee88fd64e428629fe629c19b3415f2a2e779271405efba4bdff54861 Sep 08 15:34:32.707158 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.707126 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:34:32.707256 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.707189 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:34:32.707256 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:32.707216 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:34:33.503598 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:33.503562 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9f13c08-1548-4fb3-a8b3-6f384746ada3" path="/var/lib/kubelet/pods/f9f13c08-1548-4fb3-a8b3-6f384746ada3/volumes" Sep 08 15:34:33.541736 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:33.541700 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" event={"ID":"e163be32-1a6a-4e87-b09b-a054d7f4e3a9","Type":"ContainerStarted","Data":"30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842"} Sep 08 15:34:33.541911 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:33.541753 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" event={"ID":"e163be32-1a6a-4e87-b09b-a054d7f4e3a9","Type":"ContainerStarted","Data":"4a439e8bee88fd64e428629fe629c19b3415f2a2e779271405efba4bdff54861"} Sep 08 15:34:33.566098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:33.566039 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" podStartSLOduration=1.566020089 podStartE2EDuration="1.566020089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:33.565077617 +0000 UTC m=+1012.588702086" watchObservedRunningTime="2026-09-08 15:34:33.566020089 +0000 UTC m=+1012.589644544" Sep 08 15:34:34.422309 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:34.422269 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:34.428395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:34.428368 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:34.548631 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:34.548601 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:34.549724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:34.549706 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:39.710390 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:39.710351 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg"] Sep 08 15:34:39.714079 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:39.714063 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" Sep 08 15:34:39.716959 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:39.716940 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:34:39.717053 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:39.717031 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:34:39.722386 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:39.722356 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg"] Sep 08 15:34:39.832655 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:39.832608 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f45fe9a-c039-4e1d-8705-754c6dc9f7c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg\" (UID: \"6f45fe9a-c039-4e1d-8705-754c6dc9f7c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" Sep 08 15:34:39.934069 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:39.934026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f45fe9a-c039-4e1d-8705-754c6dc9f7c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg\" (UID: \"6f45fe9a-c039-4e1d-8705-754c6dc9f7c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" Sep 08 15:34:39.934771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:39.934747 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f45fe9a-c039-4e1d-8705-754c6dc9f7c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg\" (UID: \"6f45fe9a-c039-4e1d-8705-754c6dc9f7c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" Sep 08 15:34:40.027441 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:40.027351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" Sep 08 15:34:40.309778 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:40.309749 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg"] Sep 08 15:34:40.589475 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:40.589388 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" event={"ID":"6f45fe9a-c039-4e1d-8705-754c6dc9f7c9","Type":"ContainerStarted","Data":"26e16424b70bccdc0f4cbc95fb786c0e65d96a3b869a7611bb6795938e80aaec"} Sep 08 15:34:41.596411 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:41.596376 2578 generic.go:356] "Generic (PLEG): container finished" podID="6f45fe9a-c039-4e1d-8705-754c6dc9f7c9" containerID="95409ea636222f85f4eefaed0ab866145706bae4ebc460982f74969137aeab90" exitCode=0 Sep 08 15:34:41.596747 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:41.596454 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" event={"ID":"6f45fe9a-c039-4e1d-8705-754c6dc9f7c9","Type":"ContainerDied","Data":"95409ea636222f85f4eefaed0ab866145706bae4ebc460982f74969137aeab90"} Sep 08 15:34:42.742095 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:42.742061 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" Sep 08 15:34:42.865813 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:42.865718 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f45fe9a-c039-4e1d-8705-754c6dc9f7c9-maas-api-service-ca\") pod \"6f45fe9a-c039-4e1d-8705-754c6dc9f7c9\" (UID: \"6f45fe9a-c039-4e1d-8705-754c6dc9f7c9\") " Sep 08 15:34:42.866153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:42.866103 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f45fe9a-c039-4e1d-8705-754c6dc9f7c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f45fe9a-c039-4e1d-8705-754c6dc9f7c9" (UID: "6f45fe9a-c039-4e1d-8705-754c6dc9f7c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:34:42.967470 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:42.967429 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f45fe9a-c039-4e1d-8705-754c6dc9f7c9-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:34:43.611250 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:43.611214 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" Sep 08 15:34:43.611436 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:43.611257 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg" event={"ID":"6f45fe9a-c039-4e1d-8705-754c6dc9f7c9","Type":"ContainerDied","Data":"26e16424b70bccdc0f4cbc95fb786c0e65d96a3b869a7611bb6795938e80aaec"} Sep 08 15:34:43.611436 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:43.611309 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26e16424b70bccdc0f4cbc95fb786c0e65d96a3b869a7611bb6795938e80aaec" Sep 08 15:34:50.679216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:50.679178 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg"] Sep 08 15:34:50.684903 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:50.684873 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34fed8-pv5gg"] Sep 08 15:34:51.502097 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:51.502054 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f45fe9a-c039-4e1d-8705-754c6dc9f7c9" path="/var/lib/kubelet/pods/6f45fe9a-c039-4e1d-8705-754c6dc9f7c9/volumes" Sep 08 15:34:54.695235 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:54.695143 2578 generic.go:356] "Generic (PLEG): container finished" podID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerID="30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842" exitCode=0 Sep 08 15:34:54.695235 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:54.695210 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" event={"ID":"e163be32-1a6a-4e87-b09b-a054d7f4e3a9","Type":"ContainerDied","Data":"30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842"} Sep 08 15:34:54.695665 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:54.695641 2578 scope.go:117] "RemoveContainer" containerID="30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842" Sep 08 15:34:54.696713 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:54.696684 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:34:54.696773 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:54.696753 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:34:54.696809 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:54.696787 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:34:55.703891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:55.703855 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" event={"ID":"e163be32-1a6a-4e87-b09b-a054d7f4e3a9","Type":"ContainerStarted","Data":"36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04"} Sep 08 15:34:56.422528 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:56.422479 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:34:56.422714 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:56.422678 2578 patch_prober.go:28] interesting pod/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 15:34:56.422772 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:56.422743 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 15:34:57.422332 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:57.422290 2578 patch_prober.go:28] interesting pod/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 15:34:57.422737 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:57.422374 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 15:34:58.422879 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.422837 2578 patch_prober.go:28] interesting pod/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 15:34:58.423322 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.422912 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 15:34:58.431556 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.431506 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl"] Sep 08 15:34:58.432306 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.432274 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f45fe9a-c039-4e1d-8705-754c6dc9f7c9" containerName="revoke-keys" Sep 08 15:34:58.432306 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.432305 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f45fe9a-c039-4e1d-8705-754c6dc9f7c9" containerName="revoke-keys" Sep 08 15:34:58.432513 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.432433 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f45fe9a-c039-4e1d-8705-754c6dc9f7c9" containerName="revoke-keys" Sep 08 15:34:58.436384 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.436364 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" Sep 08 15:34:58.440570 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.440534 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:34:58.440839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.440817 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:34:58.455286 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.455258 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl"] Sep 08 15:34:58.542395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.542355 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08808059-3bdf-4178-ba0e-5de44494cea0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl\" (UID: \"08808059-3bdf-4178-ba0e-5de44494cea0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" Sep 08 15:34:58.643652 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.643602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08808059-3bdf-4178-ba0e-5de44494cea0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl\" (UID: \"08808059-3bdf-4178-ba0e-5de44494cea0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" Sep 08 15:34:58.645243 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.645207 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08808059-3bdf-4178-ba0e-5de44494cea0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl\" (UID: \"08808059-3bdf-4178-ba0e-5de44494cea0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" Sep 08 15:34:58.760493 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:58.760454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" Sep 08 15:34:59.048692 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:59.048639 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl"] Sep 08 15:34:59.051562 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:34:59.051531 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08808059_3bdf_4178_ba0e_5de44494cea0.slice/crio-454dea0886401f7f03304591c7f3130ebfd2d4207f0808a4b00a982ce85fd303 WatchSource:0}: Error finding container 454dea0886401f7f03304591c7f3130ebfd2d4207f0808a4b00a982ce85fd303: Status 404 returned error can't find the container with id 454dea0886401f7f03304591c7f3130ebfd2d4207f0808a4b00a982ce85fd303 Sep 08 15:34:59.423029 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:59.422994 2578 patch_prober.go:28] interesting pod/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 15:34:59.423540 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:59.423072 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 15:34:59.742594 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:59.742539 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" event={"ID":"08808059-3bdf-4178-ba0e-5de44494cea0","Type":"ContainerStarted","Data":"c17d72b8cd8c3e4c2043b46ddc7515a34e7e739ea4c34855ddf4c96323ffd386"} Sep 08 15:34:59.742817 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:59.742615 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" event={"ID":"08808059-3bdf-4178-ba0e-5de44494cea0","Type":"ContainerStarted","Data":"454dea0886401f7f03304591c7f3130ebfd2d4207f0808a4b00a982ce85fd303"} Sep 08 15:34:59.764082 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:34:59.764028 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" podStartSLOduration=1.764011477 podStartE2EDuration="1.764011477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:59.761522203 +0000 UTC m=+1038.785146702" watchObservedRunningTime="2026-09-08 15:34:59.764011477 +0000 UTC m=+1038.787635934" Sep 08 15:35:00.093831 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:35:00.093794 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08808059_3bdf_4178_ba0e_5de44494cea0.slice/crio-conmon-c17d72b8cd8c3e4c2043b46ddc7515a34e7e739ea4c34855ddf4c96323ffd386.scope\": RecentStats: unable to find data in memory cache]" Sep 08 15:35:00.422583 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:00.422493 2578 patch_prober.go:28] interesting pod/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 15:35:00.422583 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:00.422550 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 15:35:00.754871 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:00.754830 2578 generic.go:356] "Generic (PLEG): container finished" podID="08808059-3bdf-4178-ba0e-5de44494cea0" containerID="c17d72b8cd8c3e4c2043b46ddc7515a34e7e739ea4c34855ddf4c96323ffd386" exitCode=0 Sep 08 15:35:00.755296 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:00.754920 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" event={"ID":"08808059-3bdf-4178-ba0e-5de44494cea0","Type":"ContainerDied","Data":"c17d72b8cd8c3e4c2043b46ddc7515a34e7e739ea4c34855ddf4c96323ffd386"} Sep 08 15:35:01.467737 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:01.467701 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:35:01.468083 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:01.468049 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:35:01.469513 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:01.469495 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:35:01.904892 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:01.904860 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" Sep 08 15:35:02.084071 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:02.083977 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08808059-3bdf-4178-ba0e-5de44494cea0-maas-api-service-ca\") pod \"08808059-3bdf-4178-ba0e-5de44494cea0\" (UID: \"08808059-3bdf-4178-ba0e-5de44494cea0\") " Sep 08 15:35:02.084374 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:02.084352 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08808059-3bdf-4178-ba0e-5de44494cea0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08808059-3bdf-4178-ba0e-5de44494cea0" (UID: "08808059-3bdf-4178-ba0e-5de44494cea0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:35:02.185333 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:02.185296 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08808059-3bdf-4178-ba0e-5de44494cea0-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:02.769876 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:02.769844 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" Sep 08 15:35:02.770157 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:02.769889 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl" event={"ID":"08808059-3bdf-4178-ba0e-5de44494cea0","Type":"ContainerDied","Data":"454dea0886401f7f03304591c7f3130ebfd2d4207f0808a4b00a982ce85fd303"} Sep 08 15:35:02.770157 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:02.769944 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="454dea0886401f7f03304591c7f3130ebfd2d4207f0808a4b00a982ce85fd303" Sep 08 15:35:08.168915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.168872 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx"] Sep 08 15:35:08.169358 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.169334 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08808059-3bdf-4178-ba0e-5de44494cea0" containerName="revoke-keys" Sep 08 15:35:08.169358 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.169347 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="08808059-3bdf-4178-ba0e-5de44494cea0" containerName="revoke-keys" Sep 08 15:35:08.169443 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.169423 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="08808059-3bdf-4178-ba0e-5de44494cea0" containerName="revoke-keys" Sep 08 15:35:08.172695 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.172665 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" Sep 08 15:35:08.179208 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.179185 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:35:08.180079 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.180059 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:35:08.196556 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.196520 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx"] Sep 08 15:35:08.243522 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.243477 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44289b97-78ba-495f-8f94-47e3a8026793-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx\" (UID: \"44289b97-78ba-495f-8f94-47e3a8026793\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" Sep 08 15:35:08.344795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.344748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44289b97-78ba-495f-8f94-47e3a8026793-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx\" (UID: \"44289b97-78ba-495f-8f94-47e3a8026793\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" Sep 08 15:35:08.345475 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.345446 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44289b97-78ba-495f-8f94-47e3a8026793-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx\" (UID: \"44289b97-78ba-495f-8f94-47e3a8026793\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" Sep 08 15:35:08.485884 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.485848 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" Sep 08 15:35:08.790817 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.790788 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx"] Sep 08 15:35:08.792966 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:35:08.792939 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44289b97_78ba_495f_8f94_47e3a8026793.slice/crio-35be2e279d425e7ebd18cec98eedf41fbd7a91133763fa907ef743700bfd768a WatchSource:0}: Error finding container 35be2e279d425e7ebd18cec98eedf41fbd7a91133763fa907ef743700bfd768a: Status 404 returned error can't find the container with id 35be2e279d425e7ebd18cec98eedf41fbd7a91133763fa907ef743700bfd768a Sep 08 15:35:08.809053 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:08.809023 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" event={"ID":"44289b97-78ba-495f-8f94-47e3a8026793","Type":"ContainerStarted","Data":"35be2e279d425e7ebd18cec98eedf41fbd7a91133763fa907ef743700bfd768a"} Sep 08 15:35:09.459886 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:09.459850 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl"] Sep 08 15:35:09.463099 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:09.463071 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e39762-jb5kl"] Sep 08 15:35:09.502875 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:09.502832 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08808059-3bdf-4178-ba0e-5de44494cea0" path="/var/lib/kubelet/pods/08808059-3bdf-4178-ba0e-5de44494cea0/volumes" Sep 08 15:35:09.817448 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:09.817415 2578 generic.go:356] "Generic (PLEG): container finished" podID="44289b97-78ba-495f-8f94-47e3a8026793" containerID="2afc8471ffb8777e12e0ff4c28610afb109499c3fe437ea8702a18e401fcb4a0" exitCode=0 Sep 08 15:35:09.817623 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:09.817503 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" event={"ID":"44289b97-78ba-495f-8f94-47e3a8026793","Type":"ContainerDied","Data":"2afc8471ffb8777e12e0ff4c28610afb109499c3fe437ea8702a18e401fcb4a0"} Sep 08 15:35:10.975042 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:10.975014 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" Sep 08 15:35:11.071369 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:11.071322 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44289b97-78ba-495f-8f94-47e3a8026793-maas-api-service-ca\") pod \"44289b97-78ba-495f-8f94-47e3a8026793\" (UID: \"44289b97-78ba-495f-8f94-47e3a8026793\") " Sep 08 15:35:11.071711 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:11.071687 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44289b97-78ba-495f-8f94-47e3a8026793-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44289b97-78ba-495f-8f94-47e3a8026793" (UID: "44289b97-78ba-495f-8f94-47e3a8026793"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:35:11.172930 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:11.172838 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44289b97-78ba-495f-8f94-47e3a8026793-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:11.832037 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:11.831944 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" event={"ID":"44289b97-78ba-495f-8f94-47e3a8026793","Type":"ContainerDied","Data":"35be2e279d425e7ebd18cec98eedf41fbd7a91133763fa907ef743700bfd768a"} Sep 08 15:35:11.832037 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:11.831979 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx" Sep 08 15:35:11.832037 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:11.831993 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35be2e279d425e7ebd18cec98eedf41fbd7a91133763fa907ef743700bfd768a" Sep 08 15:35:16.209436 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:16.209395 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg"] Sep 08 15:35:16.210335 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:16.210271 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" containerID="cri-o://b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a" gracePeriod=30 Sep 08 15:35:19.460906 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:19.460862 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx"] Sep 08 15:35:19.465536 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:19.465504 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9fbfa439-dtzfx"] Sep 08 15:35:19.501842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:19.501804 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44289b97-78ba-495f-8f94-47e3a8026793" path="/var/lib/kubelet/pods/44289b97-78ba-495f-8f94-47e3a8026793/volumes" Sep 08 15:35:21.475887 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.475814 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:35:21.578422 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578390 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-certs\") pod \"d7d9e082-b577-45de-b553-ba564c53c1b8\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " Sep 08 15:35:21.578620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578433 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-token\") pod \"d7d9e082-b577-45de-b553-ba564c53c1b8\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " Sep 08 15:35:21.578620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578458 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-podinfo\") pod \"d7d9e082-b577-45de-b553-ba564c53c1b8\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " Sep 08 15:35:21.578620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578514 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-socket\") pod \"d7d9e082-b577-45de-b553-ba564c53c1b8\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " Sep 08 15:35:21.578620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578536 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-credential-socket\") pod \"d7d9e082-b577-45de-b553-ba564c53c1b8\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " Sep 08 15:35:21.578620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578593 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7d9e082-b577-45de-b553-ba564c53c1b8-istiod-ca-cert\") pod \"d7d9e082-b577-45de-b553-ba564c53c1b8\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " Sep 08 15:35:21.579303 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578639 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz8m2\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-kube-api-access-qz8m2\") pod \"d7d9e082-b577-45de-b553-ba564c53c1b8\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " Sep 08 15:35:21.579303 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578880 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-data\") pod \"d7d9e082-b577-45de-b553-ba564c53c1b8\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " Sep 08 15:35:21.579303 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578968 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-envoy\") pod \"d7d9e082-b577-45de-b553-ba564c53c1b8\" (UID: \"d7d9e082-b577-45de-b553-ba564c53c1b8\") " Sep 08 15:35:21.579303 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578877 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7d9e082-b577-45de-b553-ba564c53c1b8" (UID: "d7d9e082-b577-45de-b553-ba564c53c1b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:21.579303 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.579097 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7d9e082-b577-45de-b553-ba564c53c1b8" (UID: "d7d9e082-b577-45de-b553-ba564c53c1b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:21.579552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.579298 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7d9e082-b577-45de-b553-ba564c53c1b8" (UID: "d7d9e082-b577-45de-b553-ba564c53c1b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:21.579552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.578930 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7d9e082-b577-45de-b553-ba564c53c1b8" (UID: "d7d9e082-b577-45de-b553-ba564c53c1b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:21.579552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.579486 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7d9e082-b577-45de-b553-ba564c53c1b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7d9e082-b577-45de-b553-ba564c53c1b8" (UID: "d7d9e082-b577-45de-b553-ba564c53c1b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:35:21.579667 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.579631 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:21.579667 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.579651 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:21.579667 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.579666 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:21.579818 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.579679 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:21.579818 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.579693 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7d9e082-b577-45de-b553-ba564c53c1b8-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:21.580942 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.580919 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7d9e082-b577-45de-b553-ba564c53c1b8" (UID: "d7d9e082-b577-45de-b553-ba564c53c1b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:35:21.581356 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.581340 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7d9e082-b577-45de-b553-ba564c53c1b8" (UID: "d7d9e082-b577-45de-b553-ba564c53c1b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:21.581511 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.581496 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7d9e082-b577-45de-b553-ba564c53c1b8" (UID: "d7d9e082-b577-45de-b553-ba564c53c1b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:35:21.581706 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.581691 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-kube-api-access-qz8m2" (OuterVolumeSpecName: "kube-api-access-qz8m2") pod "d7d9e082-b577-45de-b553-ba564c53c1b8" (UID: "d7d9e082-b577-45de-b553-ba564c53c1b8"). InnerVolumeSpecName "kube-api-access-qz8m2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:35:21.680679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.680639 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:21.680679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.680672 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:21.680679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.680683 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7d9e082-b577-45de-b553-ba564c53c1b8-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:21.680679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.680691 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz8m2\" (UniqueName: \"kubernetes.io/projected/d7d9e082-b577-45de-b553-ba564c53c1b8-kube-api-access-qz8m2\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:21.900615 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.900512 2578 generic.go:356] "Generic (PLEG): container finished" podID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerID="b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a" exitCode=0 Sep 08 15:35:21.900615 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.900550 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" event={"ID":"d7d9e082-b577-45de-b553-ba564c53c1b8","Type":"ContainerDied","Data":"b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a"} Sep 08 15:35:21.900615 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.900570 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" Sep 08 15:35:21.900615 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.900608 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg" event={"ID":"d7d9e082-b577-45de-b553-ba564c53c1b8","Type":"ContainerDied","Data":"76e2de02ad686aada44ae6237e0ef329dbb9afa86d24469688b06336e85e79e4"} Sep 08 15:35:21.900948 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.900635 2578 scope.go:117] "RemoveContainer" containerID="b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a" Sep 08 15:35:21.915407 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.915381 2578 scope.go:117] "RemoveContainer" containerID="af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282" Sep 08 15:35:21.927272 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.927245 2578 scope.go:117] "RemoveContainer" containerID="b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a" Sep 08 15:35:21.927705 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:35:21.927661 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a\": container with ID starting with b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a not found: ID does not exist" containerID="b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a" Sep 08 15:35:21.927819 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.927718 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a"} err="failed to get container status \"b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a\": rpc error: code = NotFound desc = could not find container \"b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a\": container with ID starting with b0eb84b7e49e3a4b8f78fa0be439cf7ae2bc1bc8c5fcc0b4cb4fb34bebf9dc8a not found: ID does not exist" Sep 08 15:35:21.927819 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.927745 2578 scope.go:117] "RemoveContainer" containerID="af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282" Sep 08 15:35:21.928185 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:35:21.928159 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282\": container with ID starting with af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282 not found: ID does not exist" containerID="af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282" Sep 08 15:35:21.928253 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.928197 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282"} err="failed to get container status \"af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282\": rpc error: code = NotFound desc = could not find container \"af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282\": container with ID starting with af655a2430f766933724c8a72d22666c32342eaff3df89130397923fa0ae7282 not found: ID does not exist" Sep 08 15:35:21.930010 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.929990 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg"] Sep 08 15:35:21.939283 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:21.939251 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e39762-openshift-default-7b5dc6bbd4-kbtcg"] Sep 08 15:35:23.502223 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:23.502187 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" path="/var/lib/kubelet/pods/d7d9e082-b577-45de-b553-ba564c53c1b8/volumes" Sep 08 15:35:26.664362 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:26.664319 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4"] Sep 08 15:35:26.664906 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:26.664548 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" containerID="cri-o://36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04" gracePeriod=30 Sep 08 15:35:31.470424 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.470385 2578 patch_prober.go:28] interesting pod/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 15:35:31.470798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.470459 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 15:35:31.916379 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.916348 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:35:31.975829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.975792 2578 generic.go:356] "Generic (PLEG): container finished" podID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerID="36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04" exitCode=0 Sep 08 15:35:31.975999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.975840 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" Sep 08 15:35:31.975999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.975877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" event={"ID":"e163be32-1a6a-4e87-b09b-a054d7f4e3a9","Type":"ContainerDied","Data":"36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04"} Sep 08 15:35:31.975999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.975934 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4" event={"ID":"e163be32-1a6a-4e87-b09b-a054d7f4e3a9","Type":"ContainerDied","Data":"4a439e8bee88fd64e428629fe629c19b3415f2a2e779271405efba4bdff54861"} Sep 08 15:35:31.975999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.975962 2578 scope.go:117] "RemoveContainer" containerID="36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04" Sep 08 15:35:31.986224 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.986202 2578 scope.go:117] "RemoveContainer" containerID="30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842" Sep 08 15:35:31.995691 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.995669 2578 scope.go:117] "RemoveContainer" containerID="36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04" Sep 08 15:35:31.996054 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:35:31.996033 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04\": container with ID starting with 36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04 not found: ID does not exist" containerID="36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04" Sep 08 15:35:31.996158 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.996066 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04"} err="failed to get container status \"36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04\": rpc error: code = NotFound desc = could not find container \"36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04\": container with ID starting with 36097bca8533818a212895856843959aa53527d29e66fb7473847b386e501f04 not found: ID does not exist" Sep 08 15:35:31.996158 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.996084 2578 scope.go:117] "RemoveContainer" containerID="30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842" Sep 08 15:35:31.996406 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:35:31.996384 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842\": container with ID starting with 30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842 not found: ID does not exist" containerID="30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842" Sep 08 15:35:31.996451 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:31.996411 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842"} err="failed to get container status \"30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842\": rpc error: code = NotFound desc = could not find container \"30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842\": container with ID starting with 30656625b5c9ca315d9f0bba8a6013af959c41aecad3fedf21510a2569dde842 not found: ID does not exist" Sep 08 15:35:32.088057 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.087964 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-socket\") pod \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " Sep 08 15:35:32.088057 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088014 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-token\") pod \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " Sep 08 15:35:32.088057 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088055 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqxvt\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-kube-api-access-lqxvt\") pod \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " Sep 08 15:35:32.088370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088082 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-data\") pod \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " Sep 08 15:35:32.088370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088151 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-envoy\") pod \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " Sep 08 15:35:32.088370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088197 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-certs\") pod \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " Sep 08 15:35:32.088370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088221 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-podinfo\") pod \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " Sep 08 15:35:32.088370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088276 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-credential-socket\") pod \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " Sep 08 15:35:32.088370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088304 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istiod-ca-cert\") pod \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\" (UID: \"e163be32-1a6a-4e87-b09b-a054d7f4e3a9\") " Sep 08 15:35:32.088370 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088300 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e163be32-1a6a-4e87-b09b-a054d7f4e3a9" (UID: "e163be32-1a6a-4e87-b09b-a054d7f4e3a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:32.088733 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088523 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "e163be32-1a6a-4e87-b09b-a054d7f4e3a9" (UID: "e163be32-1a6a-4e87-b09b-a054d7f4e3a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:32.088733 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088563 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:32.088733 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088596 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e163be32-1a6a-4e87-b09b-a054d7f4e3a9" (UID: "e163be32-1a6a-4e87-b09b-a054d7f4e3a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:32.089058 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088758 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e163be32-1a6a-4e87-b09b-a054d7f4e3a9" (UID: "e163be32-1a6a-4e87-b09b-a054d7f4e3a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:32.089058 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.088861 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e163be32-1a6a-4e87-b09b-a054d7f4e3a9" (UID: "e163be32-1a6a-4e87-b09b-a054d7f4e3a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:35:32.090610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.090570 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e163be32-1a6a-4e87-b09b-a054d7f4e3a9" (UID: "e163be32-1a6a-4e87-b09b-a054d7f4e3a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:35:32.090747 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.090673 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-kube-api-access-lqxvt" (OuterVolumeSpecName: "kube-api-access-lqxvt") pod "e163be32-1a6a-4e87-b09b-a054d7f4e3a9" (UID: "e163be32-1a6a-4e87-b09b-a054d7f4e3a9"). InnerVolumeSpecName "kube-api-access-lqxvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:35:32.090747 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.090682 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "e163be32-1a6a-4e87-b09b-a054d7f4e3a9" (UID: "e163be32-1a6a-4e87-b09b-a054d7f4e3a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:35:32.090747 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.090687 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e163be32-1a6a-4e87-b09b-a054d7f4e3a9" (UID: "e163be32-1a6a-4e87-b09b-a054d7f4e3a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:35:32.190072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.189972 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:32.190072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.190018 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:32.190072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.190029 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:32.190072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.190037 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:32.190072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.190045 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:32.190072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.190052 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:32.190072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.190060 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqxvt\" (UniqueName: \"kubernetes.io/projected/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-kube-api-access-lqxvt\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:32.190072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.190068 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e163be32-1a6a-4e87-b09b-a054d7f4e3a9-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:35:32.308383 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.308343 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4"] Sep 08 15:35:32.312030 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:32.312002 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9fbfa439-openshift-default-6bfc67ffd8-xrgv4"] Sep 08 15:35:33.501815 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:33.501782 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" path="/var/lib/kubelet/pods/e163be32-1a6a-4e87-b09b-a054d7f4e3a9/volumes" Sep 08 15:35:41.081784 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.081749 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd"] Sep 08 15:35:41.082279 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082245 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" Sep 08 15:35:41.082279 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082260 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" Sep 08 15:35:41.082279 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082267 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" Sep 08 15:35:41.082279 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082273 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082283 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082292 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082302 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082308 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082315 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44289b97-78ba-495f-8f94-47e3a8026793" containerName="revoke-keys" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082320 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44289b97-78ba-495f-8f94-47e3a8026793" containerName="revoke-keys" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082403 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082413 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082418 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="44289b97-78ba-495f-8f94-47e3a8026793" containerName="revoke-keys" Sep 08 15:35:41.082437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.082427 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d9e082-b577-45de-b553-ba564c53c1b8" containerName="istio-proxy" Sep 08 15:35:41.085455 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.085436 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.090818 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.090791 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-03e8df-openshift-default-dockercfg-46gtc\"" Sep 08 15:35:41.105694 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.105654 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd"] Sep 08 15:35:41.276162 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.276089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/770b478b-c229-436e-95b2-166c43fd3724-istiod-ca-cert\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.276162 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.276172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-certs\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.276395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.276245 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-envoy\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.276395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.276277 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/770b478b-c229-436e-95b2-166c43fd3724-istio-podinfo\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.276395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.276299 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-data\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.276395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.276351 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-socket\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.276526 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.276382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-istio-token\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.276526 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.276432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-credential-socket\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.276526 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.276464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfb8l\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-kube-api-access-nfb8l\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377053 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.376963 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-envoy\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377053 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/770b478b-c229-436e-95b2-166c43fd3724-istio-podinfo\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377053 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-data\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377053 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377049 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-socket\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377431 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-istio-token\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377431 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-credential-socket\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377431 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfb8l\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-kube-api-access-nfb8l\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377431 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/770b478b-c229-436e-95b2-166c43fd3724-istiod-ca-cert\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377431 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-certs\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377605 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377552 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-socket\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377688 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377669 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-data\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.377876 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.377849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-credential-socket\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.378029 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.378009 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-certs\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.378424 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.378395 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/770b478b-c229-436e-95b2-166c43fd3724-istiod-ca-cert\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.379680 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.379651 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-envoy\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.379942 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.379920 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/770b478b-c229-436e-95b2-166c43fd3724-istio-podinfo\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.388436 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.388412 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-istio-token\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.390586 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.390560 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfb8l\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-kube-api-access-nfb8l\") pod \"e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.399592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.399569 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-03e8df-openshift-default-dockercfg-46gtc\"" Sep 08 15:35:41.407303 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.407276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:41.695260 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.695234 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd"] Sep 08 15:35:41.696987 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:35:41.696954 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod770b478b_c229_436e_95b2_166c43fd3724.slice/crio-2eb4e40b67a581ed0c252089fa19b13b2bd609dc7fc2d3bed30b17a44cb377f5 WatchSource:0}: Error finding container 2eb4e40b67a581ed0c252089fa19b13b2bd609dc7fc2d3bed30b17a44cb377f5: Status 404 returned error can't find the container with id 2eb4e40b67a581ed0c252089fa19b13b2bd609dc7fc2d3bed30b17a44cb377f5 Sep 08 15:35:41.699187 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.699155 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:35:41.699288 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.699261 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:35:41.699354 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:41.699307 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:35:42.042717 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:42.042681 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" event={"ID":"770b478b-c229-436e-95b2-166c43fd3724","Type":"ContainerStarted","Data":"a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8"} Sep 08 15:35:42.042950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:42.042737 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" event={"ID":"770b478b-c229-436e-95b2-166c43fd3724","Type":"ContainerStarted","Data":"2eb4e40b67a581ed0c252089fa19b13b2bd609dc7fc2d3bed30b17a44cb377f5"} Sep 08 15:35:42.074425 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:42.074368 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" podStartSLOduration=1.07435166 podStartE2EDuration="1.07435166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:42.071197768 +0000 UTC m=+1081.094822223" watchObservedRunningTime="2026-09-08 15:35:42.07435166 +0000 UTC m=+1081.097976115" Sep 08 15:35:42.408255 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:42.408165 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:42.413209 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:42.413175 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:43.049475 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:43.049443 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:43.050556 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:43.050526 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:35:45.981017 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:45.980978 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r"] Sep 08 15:35:45.981546 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:45.981530 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e163be32-1a6a-4e87-b09b-a054d7f4e3a9" containerName="istio-proxy" Sep 08 15:35:45.984907 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:45.984883 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:45.988261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:45.988235 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-9fb31f-openshift-default-dockercfg-zh62f\"" Sep 08 15:35:46.003810 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.003775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r"] Sep 08 15:35:46.020312 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.020274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a01f99c-66cf-45e5-808c-f24035420c28-istiod-ca-cert\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.020503 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.020321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-istio-token\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.020503 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.020346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a01f99c-66cf-45e5-808c-f24035420c28-istio-podinfo\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.020503 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.020407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-socket\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.020503 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.020452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-envoy\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.020503 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.020476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-credential-socket\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.020698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.020522 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlmtx\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-kube-api-access-jlmtx\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.020698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.020595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-certs\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.020698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.020644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-data\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121380 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-certs\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121589 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121392 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-data\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121589 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a01f99c-66cf-45e5-808c-f24035420c28-istiod-ca-cert\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121589 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-istio-token\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121589 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121497 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a01f99c-66cf-45e5-808c-f24035420c28-istio-podinfo\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121589 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-socket\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121589 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-envoy\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-credential-socket\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlmtx\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-kube-api-access-jlmtx\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.121913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-data\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.122052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.121934 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-socket\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.122052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.122015 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-certs\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.122157 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.122136 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-credential-socket\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.122249 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.122230 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a01f99c-66cf-45e5-808c-f24035420c28-istiod-ca-cert\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.124168 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.124105 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-envoy\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.124299 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.124280 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a01f99c-66cf-45e5-808c-f24035420c28-istio-podinfo\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.139081 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.139046 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-istio-token\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.139245 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.139141 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlmtx\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-kube-api-access-jlmtx\") pod \"e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.299765 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.299673 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:46.583330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.583303 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r"] Sep 08 15:35:46.584811 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:35:46.584786 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a01f99c_66cf_45e5_808c_f24035420c28.slice/crio-8518a3035336fadcb836c7f8628ddf86b9f90a4b473dc850a43f01f5837d94b3 WatchSource:0}: Error finding container 8518a3035336fadcb836c7f8628ddf86b9f90a4b473dc850a43f01f5837d94b3: Status 404 returned error can't find the container with id 8518a3035336fadcb836c7f8628ddf86b9f90a4b473dc850a43f01f5837d94b3 Sep 08 15:35:46.586988 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.586953 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:35:46.587086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.587033 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:35:46.587086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:46.587078 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:35:47.077983 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:47.077946 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" event={"ID":"6a01f99c-66cf-45e5-808c-f24035420c28","Type":"ContainerStarted","Data":"125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce"} Sep 08 15:35:47.078491 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:47.078005 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" event={"ID":"6a01f99c-66cf-45e5-808c-f24035420c28","Type":"ContainerStarted","Data":"8518a3035336fadcb836c7f8628ddf86b9f90a4b473dc850a43f01f5837d94b3"} Sep 08 15:35:47.115923 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:47.115867 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podStartSLOduration=2.115849831 podStartE2EDuration="2.115849831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:47.115795197 +0000 UTC m=+1086.139419652" watchObservedRunningTime="2026-09-08 15:35:47.115849831 +0000 UTC m=+1086.139474286" Sep 08 15:35:47.300839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:47.300799 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:47.329649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:47.329569 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:48.084036 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:48.084004 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:48.085157 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:48.085136 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:35:59.091337 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:59.091274 2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 08 15:35:59.091815 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:35:59.091363 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 15:36:02.208013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:02.207980 2578 generic.go:356] "Generic (PLEG): container finished" podID="770b478b-c229-436e-95b2-166c43fd3724" containerID="a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8" exitCode=0 Sep 08 15:36:02.208449 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:02.208052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" event={"ID":"770b478b-c229-436e-95b2-166c43fd3724","Type":"ContainerDied","Data":"a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8"} Sep 08 15:36:02.208509 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:02.208496 2578 scope.go:117] "RemoveContainer" containerID="a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8" Sep 08 15:36:02.209544 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:02.209519 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:02.209603 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:02.209582 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:02.209640 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:02.209614 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:03.219143 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:03.219089 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" event={"ID":"770b478b-c229-436e-95b2-166c43fd3724","Type":"ContainerStarted","Data":"82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797"} Sep 08 15:36:03.408331 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:03.408294 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:36:03.408523 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:03.408482 2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Sep 08 15:36:03.408598 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:03.408537 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Sep 08 15:36:04.125664 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:04.125304 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 08 15:36:04.125664 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:04.125381 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 15:36:04.408185 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:04.408066 2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Sep 08 15:36:04.408185 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:04.408169 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Sep 08 15:36:05.408336 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:05.408298 2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Sep 08 15:36:05.408740 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:05.408361 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Sep 08 15:36:06.265040 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:06.265001 2578 generic.go:356] "Generic (PLEG): container finished" podID="6a01f99c-66cf-45e5-808c-f24035420c28" containerID="125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce" exitCode=0 Sep 08 15:36:06.265278 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:06.265070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" event={"ID":"6a01f99c-66cf-45e5-808c-f24035420c28","Type":"ContainerDied","Data":"125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce"} Sep 08 15:36:06.265544 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:06.265526 2578 scope.go:117] "RemoveContainer" containerID="125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce" Sep 08 15:36:06.266741 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:06.266714 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:06.266820 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:06.266784 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:06.266820 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:06.266816 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:06.408317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:06.408281 2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Sep 08 15:36:06.408483 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:06.408349 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Sep 08 15:36:07.274745 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:07.274704 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" event={"ID":"6a01f99c-66cf-45e5-808c-f24035420c28","Type":"ContainerStarted","Data":"0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36"} Sep 08 15:36:07.300515 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:07.300475 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:36:07.300725 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:07.300669 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 15:36:07.300795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:07.300728 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 15:36:07.408557 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:07.408516 2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Sep 08 15:36:07.408957 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:07.408574 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Sep 08 15:36:08.300627 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:08.300585 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 15:36:08.300910 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:08.300667 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 15:36:08.548934 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:08.548901 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:36:08.550736 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:08.550645 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:36:08.557445 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:08.557410 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:36:09.300204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.300166 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 15:36:09.300431 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.300233 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 15:36:09.557246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.557140 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6"] Sep 08 15:36:09.561602 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.561563 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6"] Sep 08 15:36:09.561782 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.561695 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.564842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.564808 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-a7ae4e-openshift-default-dockercfg-572fc\"" Sep 08 15:36:09.684973 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.684930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-envoy\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.685178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.684981 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-socket\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.685178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.685001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-data\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.685178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.685020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-certs\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.685178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.685048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-token\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.685178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.685074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-credential-socket\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.685178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.685103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75bb71b2-38c0-4247-b4c4-357feec5c833-istiod-ca-cert\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.685402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.685246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfwzq\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-kube-api-access-lfwzq\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.685402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.685289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-podinfo\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786285 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786249 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-certs\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786285 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-token\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-credential-socket\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786339 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75bb71b2-38c0-4247-b4c4-357feec5c833-istiod-ca-cert\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfwzq\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-kube-api-access-lfwzq\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-podinfo\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786473 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-envoy\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-socket\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786538 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-data\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786910 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786756 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-certs\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786910 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786771 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-credential-socket\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.786910 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786881 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-data\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.787036 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.786970 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-socket\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.787147 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.787103 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75bb71b2-38c0-4247-b4c4-357feec5c833-istiod-ca-cert\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.789248 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.789216 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-envoy\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.789458 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.789440 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-podinfo\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.795620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.795593 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-token\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.795728 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.795670 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfwzq\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-kube-api-access-lfwzq\") pod \"e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:09.874256 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:09.874163 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:10.185426 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.185396 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6"] Sep 08 15:36:10.187247 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:36:10.187217 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75bb71b2_38c0_4247_b4c4_357feec5c833.slice/crio-2c4ca05c3f7e3e3a1897cee3994444bebd190fff3fdb76e76d58b9d6e2924e7e WatchSource:0}: Error finding container 2c4ca05c3f7e3e3a1897cee3994444bebd190fff3fdb76e76d58b9d6e2924e7e: Status 404 returned error can't find the container with id 2c4ca05c3f7e3e3a1897cee3994444bebd190fff3fdb76e76d58b9d6e2924e7e Sep 08 15:36:10.189681 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.189648 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:10.189785 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.189717 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:10.189785 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.189745 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:10.300072 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.300039 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 15:36:10.300272 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.300097 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 15:36:10.301861 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.301817 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" event={"ID":"75bb71b2-38c0-4247-b4c4-357feec5c833","Type":"ContainerStarted","Data":"84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2"} Sep 08 15:36:10.302824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.302787 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" event={"ID":"75bb71b2-38c0-4247-b4c4-357feec5c833","Type":"ContainerStarted","Data":"2c4ca05c3f7e3e3a1897cee3994444bebd190fff3fdb76e76d58b9d6e2924e7e"} Sep 08 15:36:10.339140 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.335560 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" podStartSLOduration=1.335536597 podStartE2EDuration="1.335536597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:10.335459631 +0000 UTC m=+1109.359084089" watchObservedRunningTime="2026-09-08 15:36:10.335536597 +0000 UTC m=+1109.359161057" Sep 08 15:36:10.874753 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:10.874715 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:11.300875 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:11.300832 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 15:36:11.301068 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:11.300912 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 15:36:11.879953 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:11.879920 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:12.300978 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:12.300937 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 15:36:12.301225 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:12.301013 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 15:36:12.314021 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:12.313990 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:12.314925 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:12.314906 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:13.300540 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:13.300506 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 15:36:13.300997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:13.300572 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 15:36:14.303773 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:14.303738 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:36:14.304183 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:14.304030 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:36:14.304811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:14.304796 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:36:28.356315 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:28.356268 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 08 15:36:28.356849 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:28.356354 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 15:36:30.441182 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:30.441144 2578 generic.go:356] "Generic (PLEG): container finished" podID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerID="84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2" exitCode=0 Sep 08 15:36:30.441182 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:30.441184 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" event={"ID":"75bb71b2-38c0-4247-b4c4-357feec5c833","Type":"ContainerDied","Data":"84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2"} Sep 08 15:36:30.441638 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:30.441619 2578 scope.go:117] "RemoveContainer" containerID="84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2" Sep 08 15:36:30.442815 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:30.442788 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:30.442920 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:30.442906 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:30.442957 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:30.442944 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:31.452927 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:31.452896 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" event={"ID":"75bb71b2-38c0-4247-b4c4-357feec5c833","Type":"ContainerStarted","Data":"9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88"} Sep 08 15:36:31.875137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:31.875013 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:31.875299 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:31.875266 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 15:36:31.875342 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:31.875321 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 15:36:32.875198 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:32.875156 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 15:36:32.875593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:32.875238 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 15:36:33.874766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:33.874725 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 15:36:33.874992 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:33.874790 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 15:36:34.874752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:34.874712 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 15:36:34.875180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:34.874769 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 15:36:35.875483 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:35.875446 2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 15:36:35.875884 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:35.875504 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 15:36:36.878500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:36.878459 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:36.878956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:36.878756 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:36.879655 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:36.879635 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:36:37.979441 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:37.979400 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7749-d807cd"] Sep 08 15:36:37.983222 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:37.983200 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-d807cd" Sep 08 15:36:37.986315 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:37.986283 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-md8bz\"" Sep 08 15:36:37.992281 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:37.992250 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-d807cd"] Sep 08 15:36:38.087159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:38.087087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb4z8\" (UniqueName: \"kubernetes.io/projected/cb3d9972-8c2d-42c5-92ec-6f1f91096011-kube-api-access-cb4z8\") pod \"test-curl-7749-d807cd\" (UID: \"cb3d9972-8c2d-42c5-92ec-6f1f91096011\") " pod="opendatahub/test-curl-7749-d807cd" Sep 08 15:36:38.188488 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:38.188446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb4z8\" (UniqueName: \"kubernetes.io/projected/cb3d9972-8c2d-42c5-92ec-6f1f91096011-kube-api-access-cb4z8\") pod \"test-curl-7749-d807cd\" (UID: \"cb3d9972-8c2d-42c5-92ec-6f1f91096011\") " pod="opendatahub/test-curl-7749-d807cd" Sep 08 15:36:38.199233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:38.199202 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb4z8\" (UniqueName: \"kubernetes.io/projected/cb3d9972-8c2d-42c5-92ec-6f1f91096011-kube-api-access-cb4z8\") pod \"test-curl-7749-d807cd\" (UID: \"cb3d9972-8c2d-42c5-92ec-6f1f91096011\") " pod="opendatahub/test-curl-7749-d807cd" Sep 08 15:36:38.294364 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:38.294270 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-d807cd" Sep 08 15:36:38.569706 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:38.569673 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-d807cd"] Sep 08 15:36:38.571313 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:36:38.571280 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb3d9972_8c2d_42c5_92ec_6f1f91096011.slice/crio-af5054096c39cff293e1e98ad492be49ec5a39005bff53f535d7f91e7ab2b891 WatchSource:0}: Error finding container af5054096c39cff293e1e98ad492be49ec5a39005bff53f535d7f91e7ab2b891: Status 404 returned error can't find the container with id af5054096c39cff293e1e98ad492be49ec5a39005bff53f535d7f91e7ab2b891 Sep 08 15:36:38.573700 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:38.573676 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 15:36:39.507583 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:39.507538 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-d807cd" event={"ID":"cb3d9972-8c2d-42c5-92ec-6f1f91096011","Type":"ContainerStarted","Data":"af5054096c39cff293e1e98ad492be49ec5a39005bff53f535d7f91e7ab2b891"} Sep 08 15:36:41.524667 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:41.524632 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-d807cd" event={"ID":"cb3d9972-8c2d-42c5-92ec-6f1f91096011","Type":"ContainerStarted","Data":"d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d"} Sep 08 15:36:41.543531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:41.543477 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7749-d807cd" podStartSLOduration=2.443105359 podStartE2EDuration="4.543462062s" totalImagesPullingTime="2.100356703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:41.542993935 +0000 UTC m=+1140.566618391" watchObservedRunningTime="2026-09-08 15:36:41.543462062 +0000 UTC m=+1140.567086518" Sep 08 15:36:42.262574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.262465 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7749-d807cd"] Sep 08 15:36:42.267269 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.267236 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7749-d807cd"] Sep 08 15:36:42.386333 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.386297 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7749-048db4"] Sep 08 15:36:42.386840 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.386826 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb3d9972-8c2d-42c5-92ec-6f1f91096011" containerName="test-curl-7749-d807cd" Sep 08 15:36:42.386887 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.386842 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb3d9972-8c2d-42c5-92ec-6f1f91096011" containerName="test-curl-7749-d807cd" Sep 08 15:36:42.386930 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.386925 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb3d9972-8c2d-42c5-92ec-6f1f91096011" containerName="test-curl-7749-d807cd" Sep 08 15:36:42.390412 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.390397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-048db4" Sep 08 15:36:42.395439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.395411 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-048db4"] Sep 08 15:36:42.531490 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:42.531392 2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7749-d807cd" container="test-curl-7749-d807cd" Sep 08 15:36:42.533069 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.533049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flrfh\" (UniqueName: \"kubernetes.io/projected/2fbb87be-8121-456c-a849-43387bc04e8f-kube-api-access-flrfh\") pod \"test-curl-7749-048db4\" (UID: \"2fbb87be-8121-456c-a849-43387bc04e8f\") " pod="opendatahub/test-curl-7749-048db4" Sep 08 15:36:42.634595 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.634550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flrfh\" (UniqueName: \"kubernetes.io/projected/2fbb87be-8121-456c-a849-43387bc04e8f-kube-api-access-flrfh\") pod \"test-curl-7749-048db4\" (UID: \"2fbb87be-8121-456c-a849-43387bc04e8f\") " pod="opendatahub/test-curl-7749-048db4" Sep 08 15:36:42.645044 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.645010 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flrfh\" (UniqueName: \"kubernetes.io/projected/2fbb87be-8121-456c-a849-43387bc04e8f-kube-api-access-flrfh\") pod \"test-curl-7749-048db4\" (UID: \"2fbb87be-8121-456c-a849-43387bc04e8f\") " pod="opendatahub/test-curl-7749-048db4" Sep 08 15:36:42.702210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.702164 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-048db4" Sep 08 15:36:42.976172 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:42.976148 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-048db4"] Sep 08 15:36:42.978026 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:36:42.977992 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fbb87be_8121_456c_a849_43387bc04e8f.slice/crio-e11cfac719dfd068db18d8e20aeb3c6278bcf72bf44731257531b9dec7e834ef WatchSource:0}: Error finding container e11cfac719dfd068db18d8e20aeb3c6278bcf72bf44731257531b9dec7e834ef: Status 404 returned error can't find the container with id e11cfac719dfd068db18d8e20aeb3c6278bcf72bf44731257531b9dec7e834ef Sep 08 15:36:43.541457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:43.541413 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-048db4" event={"ID":"2fbb87be-8121-456c-a849-43387bc04e8f","Type":"ContainerStarted","Data":"ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31"} Sep 08 15:36:43.541841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:43.541469 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-048db4" event={"ID":"2fbb87be-8121-456c-a849-43387bc04e8f","Type":"ContainerStarted","Data":"e11cfac719dfd068db18d8e20aeb3c6278bcf72bf44731257531b9dec7e834ef"} Sep 08 15:36:43.541841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:43.541598 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7749-d807cd" podUID="cb3d9972-8c2d-42c5-92ec-6f1f91096011" containerName="test-curl-7749-d807cd" containerID="cri-o://d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d" gracePeriod=2 Sep 08 15:36:43.559726 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:43.559667 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7749-048db4" podStartSLOduration=1.559646654 podStartE2EDuration="1.559646654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:43.557695275 +0000 UTC m=+1142.581319735" watchObservedRunningTime="2026-09-08 15:36:43.559646654 +0000 UTC m=+1142.583271110" Sep 08 15:36:44.187383 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.187344 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7749-048db4"] Sep 08 15:36:44.192210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.192126 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7749-048db4"] Sep 08 15:36:44.310816 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.310783 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7749-8d67b0"] Sep 08 15:36:44.311245 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.311231 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fbb87be-8121-456c-a849-43387bc04e8f" containerName="test-curl-7749-048db4" Sep 08 15:36:44.311310 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.311246 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fbb87be-8121-456c-a849-43387bc04e8f" containerName="test-curl-7749-048db4" Sep 08 15:36:44.311352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.311330 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fbb87be-8121-456c-a849-43387bc04e8f" containerName="test-curl-7749-048db4" Sep 08 15:36:44.314534 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.314519 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-8d67b0" Sep 08 15:36:44.331128 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.331071 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-8d67b0"] Sep 08 15:36:44.452408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.452368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88hks\" (UniqueName: \"kubernetes.io/projected/683fbd8b-5468-4d64-abeb-974eac1c167c-kube-api-access-88hks\") pod \"test-curl-7749-8d67b0\" (UID: \"683fbd8b-5468-4d64-abeb-974eac1c167c\") " pod="opendatahub/test-curl-7749-8d67b0" Sep 08 15:36:44.548467 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:44.548431 2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7749-048db4" container="test-curl-7749-048db4" Sep 08 15:36:44.553342 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.553309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88hks\" (UniqueName: \"kubernetes.io/projected/683fbd8b-5468-4d64-abeb-974eac1c167c-kube-api-access-88hks\") pod \"test-curl-7749-8d67b0\" (UID: \"683fbd8b-5468-4d64-abeb-974eac1c167c\") " pod="opendatahub/test-curl-7749-8d67b0" Sep 08 15:36:44.562951 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.562919 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88hks\" (UniqueName: \"kubernetes.io/projected/683fbd8b-5468-4d64-abeb-974eac1c167c-kube-api-access-88hks\") pod \"test-curl-7749-8d67b0\" (UID: \"683fbd8b-5468-4d64-abeb-974eac1c167c\") " pod="opendatahub/test-curl-7749-8d67b0" Sep 08 15:36:44.626705 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.626667 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-8d67b0" Sep 08 15:36:44.902742 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:44.902711 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-8d67b0"] Sep 08 15:36:44.904278 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:36:44.904248 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod683fbd8b_5468_4d64_abeb_974eac1c167c.slice/crio-16f7c0f298e263b865c1cbd9ed06950471dfe0712fcb75829ebf2c7c430570ae WatchSource:0}: Error finding container 16f7c0f298e263b865c1cbd9ed06950471dfe0712fcb75829ebf2c7c430570ae: Status 404 returned error can't find the container with id 16f7c0f298e263b865c1cbd9ed06950471dfe0712fcb75829ebf2c7c430570ae Sep 08 15:36:45.557440 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:45.557407 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-8d67b0" event={"ID":"683fbd8b-5468-4d64-abeb-974eac1c167c","Type":"ContainerStarted","Data":"2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1"} Sep 08 15:36:45.557854 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:45.557455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-8d67b0" event={"ID":"683fbd8b-5468-4d64-abeb-974eac1c167c","Type":"ContainerStarted","Data":"16f7c0f298e263b865c1cbd9ed06950471dfe0712fcb75829ebf2c7c430570ae"} Sep 08 15:36:45.557854 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:45.557578 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7749-048db4" podUID="2fbb87be-8121-456c-a849-43387bc04e8f" containerName="test-curl-7749-048db4" containerID="cri-o://ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31" gracePeriod=2 Sep 08 15:36:45.574594 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:45.574542 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7749-8d67b0" podStartSLOduration=1.5745260239999999 podStartE2EDuration="1.574526024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:45.572422057 +0000 UTC m=+1144.596046515" watchObservedRunningTime="2026-09-08 15:36:45.574526024 +0000 UTC m=+1144.598150524" Sep 08 15:36:45.716999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:45.716969 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-d807cd" Sep 08 15:36:45.866428 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:45.866338 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb4z8\" (UniqueName: \"kubernetes.io/projected/cb3d9972-8c2d-42c5-92ec-6f1f91096011-kube-api-access-cb4z8\") pod \"cb3d9972-8c2d-42c5-92ec-6f1f91096011\" (UID: \"cb3d9972-8c2d-42c5-92ec-6f1f91096011\") " Sep 08 15:36:45.868727 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:45.868701 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb3d9972-8c2d-42c5-92ec-6f1f91096011-kube-api-access-cb4z8" (OuterVolumeSpecName: "kube-api-access-cb4z8") pod "cb3d9972-8c2d-42c5-92ec-6f1f91096011" (UID: "cb3d9972-8c2d-42c5-92ec-6f1f91096011"). InnerVolumeSpecName "kube-api-access-cb4z8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:36:45.967834 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:45.967799 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb4z8\" (UniqueName: \"kubernetes.io/projected/cb3d9972-8c2d-42c5-92ec-6f1f91096011-kube-api-access-cb4z8\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:36:46.189842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.189755 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7749-8d67b0"] Sep 08 15:36:46.195399 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.195367 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7749-8d67b0"] Sep 08 15:36:46.314037 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.313995 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7749-81ae5d"] Sep 08 15:36:46.314466 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.314451 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="683fbd8b-5468-4d64-abeb-974eac1c167c" containerName="test-curl-7749-8d67b0" Sep 08 15:36:46.314530 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.314467 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="683fbd8b-5468-4d64-abeb-974eac1c167c" containerName="test-curl-7749-8d67b0" Sep 08 15:36:46.314572 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.314562 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="683fbd8b-5468-4d64-abeb-974eac1c167c" containerName="test-curl-7749-8d67b0" Sep 08 15:36:46.317732 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.317716 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:46.326609 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.326580 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-81ae5d"] Sep 08 15:36:46.473256 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.473219 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk9sw\" (UniqueName: \"kubernetes.io/projected/e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d-kube-api-access-gk9sw\") pod \"test-curl-7749-81ae5d\" (UID: \"e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d\") " pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:46.564684 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.564648 2578 generic.go:356] "Generic (PLEG): container finished" podID="cb3d9972-8c2d-42c5-92ec-6f1f91096011" containerID="d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d" exitCode=137 Sep 08 15:36:46.565161 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.564692 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-d807cd" Sep 08 15:36:46.565161 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.564746 2578 scope.go:117] "RemoveContainer" containerID="d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d" Sep 08 15:36:46.565161 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:46.564935 2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7749-8d67b0" container="test-curl-7749-8d67b0" Sep 08 15:36:46.573943 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.573921 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk9sw\" (UniqueName: \"kubernetes.io/projected/e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d-kube-api-access-gk9sw\") pod \"test-curl-7749-81ae5d\" (UID: \"e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d\") " pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:46.574771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.574751 2578 scope.go:117] "RemoveContainer" containerID="d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d" Sep 08 15:36:46.575080 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:46.575051 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-d807cd\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="cb3d9972-8c2d-42c5-92ec-6f1f91096011" pod="opendatahub/test-curl-7749-d807cd" Sep 08 15:36:46.575176 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:46.575084 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d\": container with ID starting with d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d not found: ID does not exist" containerID="d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d" Sep 08 15:36:46.575176 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.575134 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d"} err="failed to get container status \"d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d\": rpc error: code = NotFound desc = could not find container \"d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d\": container with ID starting with d8aa992a6f8d01da87d8887e1c6527ae62a51f3695190c9659f8ba5498da9f4d not found: ID does not exist" Sep 08 15:36:46.584337 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.584318 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk9sw\" (UniqueName: \"kubernetes.io/projected/e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d-kube-api-access-gk9sw\") pod \"test-curl-7749-81ae5d\" (UID: \"e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d\") " pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:46.629155 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.629099 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:46.905355 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:46.905326 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-81ae5d"] Sep 08 15:36:46.906290 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:36:46.906267 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1ab9a69_42a2_4da3_8f0e_ef0d6f3b8e9d.slice/crio-85b9266d03354a61b3d8835ffae5f9cbc689dc3559f9549bc87904088a541bfc WatchSource:0}: Error finding container 85b9266d03354a61b3d8835ffae5f9cbc689dc3559f9549bc87904088a541bfc: Status 404 returned error can't find the container with id 85b9266d03354a61b3d8835ffae5f9cbc689dc3559f9549bc87904088a541bfc Sep 08 15:36:47.504019 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:47.503979 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb3d9972-8c2d-42c5-92ec-6f1f91096011" path="/var/lib/kubelet/pods/cb3d9972-8c2d-42c5-92ec-6f1f91096011/volumes" Sep 08 15:36:47.578950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:47.578901 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-81ae5d" event={"ID":"e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d","Type":"ContainerStarted","Data":"71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74"} Sep 08 15:36:47.579391 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:47.578955 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-81ae5d" event={"ID":"e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d","Type":"ContainerStarted","Data":"85b9266d03354a61b3d8835ffae5f9cbc689dc3559f9549bc87904088a541bfc"} Sep 08 15:36:47.579391 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:47.578992 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7749-8d67b0" podUID="683fbd8b-5468-4d64-abeb-974eac1c167c" containerName="test-curl-7749-8d67b0" containerID="cri-o://2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1" gracePeriod=2 Sep 08 15:36:47.598929 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:47.598862 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7749-81ae5d" podStartSLOduration=1.5988459769999999 podStartE2EDuration="1.598845977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:47.598607659 +0000 UTC m=+1146.622232107" watchObservedRunningTime="2026-09-08 15:36:47.598845977 +0000 UTC m=+1146.622470434" Sep 08 15:36:47.772333 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:47.772255 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-048db4" Sep 08 15:36:47.888590 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:47.888556 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flrfh\" (UniqueName: \"kubernetes.io/projected/2fbb87be-8121-456c-a849-43387bc04e8f-kube-api-access-flrfh\") pod \"2fbb87be-8121-456c-a849-43387bc04e8f\" (UID: \"2fbb87be-8121-456c-a849-43387bc04e8f\") " Sep 08 15:36:47.890850 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:47.890820 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fbb87be-8121-456c-a849-43387bc04e8f-kube-api-access-flrfh" (OuterVolumeSpecName: "kube-api-access-flrfh") pod "2fbb87be-8121-456c-a849-43387bc04e8f" (UID: "2fbb87be-8121-456c-a849-43387bc04e8f"). InnerVolumeSpecName "kube-api-access-flrfh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:36:47.991000 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:47.990966 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flrfh\" (UniqueName: \"kubernetes.io/projected/2fbb87be-8121-456c-a849-43387bc04e8f-kube-api-access-flrfh\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:36:48.239594 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:48.239548 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7749-81ae5d"] Sep 08 15:36:48.244411 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:48.244366 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7749-81ae5d"] Sep 08 15:36:48.586385 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:48.586300 2578 generic.go:356] "Generic (PLEG): container finished" podID="2fbb87be-8121-456c-a849-43387bc04e8f" containerID="ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31" exitCode=137 Sep 08 15:36:48.586385 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:48.586343 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-048db4" Sep 08 15:36:48.586885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:48.586395 2578 scope.go:117] "RemoveContainer" containerID="ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31" Sep 08 15:36:48.586885 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:48.586621 2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7749-81ae5d" container="test-curl-7749-81ae5d" Sep 08 15:36:48.596546 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:48.596526 2578 scope.go:117] "RemoveContainer" containerID="ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31" Sep 08 15:36:48.596854 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:48.596826 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-048db4\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="2fbb87be-8121-456c-a849-43387bc04e8f" pod="opendatahub/test-curl-7749-048db4" Sep 08 15:36:48.596936 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:48.596864 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31\": container with ID starting with ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31 not found: ID does not exist" containerID="ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31" Sep 08 15:36:48.596936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:48.596896 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31"} err="failed to get container status \"ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31\": rpc error: code = NotFound desc = could not find container \"ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31\": container with ID starting with ebbcc4d69fb85476d5a5ebfb15ff69b49154610b1c6d374be75d8ce73991ad31 not found: ID does not exist" Sep 08 15:36:49.166556 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.166518 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g"] Sep 08 15:36:49.167039 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.167024 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" containerName="test-curl-7749-81ae5d" Sep 08 15:36:49.167086 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.167041 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" containerName="test-curl-7749-81ae5d" Sep 08 15:36:49.167156 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.167145 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" containerName="test-curl-7749-81ae5d" Sep 08 15:36:49.171792 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.171769 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.174840 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.174819 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4e499427-openshift-default-dockercfg-wvd4w\"" Sep 08 15:36:49.185710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.185676 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g"] Sep 08 15:36:49.204226 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:49.204190 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-048db4\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="2fbb87be-8121-456c-a849-43387bc04e8f" pod="opendatahub/test-curl-7749-048db4" Sep 08 15:36:49.304591 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.304553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-token\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.304591 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.304596 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wct5r\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-kube-api-access-wct5r\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.304832 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.304687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-data\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.304832 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.304757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-credential-socket\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.304832 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.304795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-socket\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.304832 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.304813 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-certs\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.304980 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.304894 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-envoy\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.304980 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.304931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-podinfo\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.304980 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.304951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istiod-ca-cert\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406028 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.405989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-data\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406048 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-credential-socket\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-socket\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-certs\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-envoy\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-podinfo\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istiod-ca-cert\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-token\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wct5r\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-kube-api-access-wct5r\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406513 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-data\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406553 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-credential-socket\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.406574 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.406568 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-certs\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.407031 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.407011 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istiod-ca-cert\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.407082 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.407053 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-socket\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.408830 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.408806 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-envoy\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.408937 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.408896 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-podinfo\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.418825 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.418761 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wct5r\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-kube-api-access-wct5r\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.418938 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.418897 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-token\") pod \"e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.483212 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.483171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:49.502834 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.502802 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fbb87be-8121-456c-a849-43387bc04e8f" path="/var/lib/kubelet/pods/2fbb87be-8121-456c-a849-43387bc04e8f/volumes" Sep 08 15:36:49.596206 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.596165 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7749-81ae5d" podUID="e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" containerName="test-curl-7749-81ae5d" containerID="cri-o://71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74" gracePeriod=2 Sep 08 15:36:49.748185 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.748157 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-8d67b0" Sep 08 15:36:49.769486 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.769457 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g"] Sep 08 15:36:49.770398 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:36:49.770376 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbb7c62_b69a_43b3_a697_bcc8338b676b.slice/crio-18f8aa6d4d04594551254084c1984858225d2a0ad3004525c0f8460844ed3a71 WatchSource:0}: Error finding container 18f8aa6d4d04594551254084c1984858225d2a0ad3004525c0f8460844ed3a71: Status 404 returned error can't find the container with id 18f8aa6d4d04594551254084c1984858225d2a0ad3004525c0f8460844ed3a71 Sep 08 15:36:49.772748 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.772721 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:49.772828 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.772781 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:49.772828 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.772812 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:36:49.911371 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.911329 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88hks\" (UniqueName: \"kubernetes.io/projected/683fbd8b-5468-4d64-abeb-974eac1c167c-kube-api-access-88hks\") pod \"683fbd8b-5468-4d64-abeb-974eac1c167c\" (UID: \"683fbd8b-5468-4d64-abeb-974eac1c167c\") " Sep 08 15:36:49.913583 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:49.913552 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683fbd8b-5468-4d64-abeb-974eac1c167c-kube-api-access-88hks" (OuterVolumeSpecName: "kube-api-access-88hks") pod "683fbd8b-5468-4d64-abeb-974eac1c167c" (UID: "683fbd8b-5468-4d64-abeb-974eac1c167c"). InnerVolumeSpecName "kube-api-access-88hks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:36:50.012699 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:50.012656 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88hks\" (UniqueName: \"kubernetes.io/projected/683fbd8b-5468-4d64-abeb-974eac1c167c-kube-api-access-88hks\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:36:50.604258 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:50.604219 2578 generic.go:356] "Generic (PLEG): container finished" podID="683fbd8b-5468-4d64-abeb-974eac1c167c" containerID="2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1" exitCode=137 Sep 08 15:36:50.604709 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:50.604267 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-8d67b0" Sep 08 15:36:50.604709 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:50.604318 2578 scope.go:117] "RemoveContainer" containerID="2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1" Sep 08 15:36:50.606261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:50.606230 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" event={"ID":"0dbb7c62-b69a-43b3-a697-bcc8338b676b","Type":"ContainerStarted","Data":"cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14"} Sep 08 15:36:50.606387 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:50.606290 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" event={"ID":"0dbb7c62-b69a-43b3-a697-bcc8338b676b","Type":"ContainerStarted","Data":"18f8aa6d4d04594551254084c1984858225d2a0ad3004525c0f8460844ed3a71"} Sep 08 15:36:50.614954 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:50.614934 2578 scope.go:117] "RemoveContainer" containerID="2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1" Sep 08 15:36:50.615287 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:50.615265 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1\": container with ID starting with 2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1 not found: ID does not exist" containerID="2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1" Sep 08 15:36:50.615357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:50.615295 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1"} err="failed to get container status \"2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1\": rpc error: code = NotFound desc = could not find container \"2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1\": container with ID starting with 2dc069bf0d865b497b784bdbedde7adbe8b7e24eee1e0a2d4d6ec25f4c6180b1 not found: ID does not exist" Sep 08 15:36:50.638880 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:50.638827 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" podStartSLOduration=1.638811231 podStartE2EDuration="1.638811231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:50.637011169 +0000 UTC m=+1149.660635627" watchObservedRunningTime="2026-09-08 15:36:50.638811231 +0000 UTC m=+1149.662435687" Sep 08 15:36:50.639797 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:50.639769 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-8d67b0\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="683fbd8b-5468-4d64-abeb-974eac1c167c" pod="opendatahub/test-curl-7749-8d67b0" Sep 08 15:36:51.483763 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:51.483720 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:51.489371 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:51.489341 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:51.501779 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:51.501752 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="683fbd8b-5468-4d64-abeb-974eac1c167c" path="/var/lib/kubelet/pods/683fbd8b-5468-4d64-abeb-974eac1c167c/volumes" Sep 08 15:36:51.517554 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:51.517523 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-8d67b0\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="683fbd8b-5468-4d64-abeb-974eac1c167c" pod="opendatahub/test-curl-7749-8d67b0" Sep 08 15:36:51.618876 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:51.618854 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:51.619808 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:51.619790 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:36:51.797447 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:51.797419 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:51.800331 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:51.800303 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-81ae5d\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:51.938790 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:51.938752 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk9sw\" (UniqueName: \"kubernetes.io/projected/e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d-kube-api-access-gk9sw\") pod \"e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d\" (UID: \"e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d\") " Sep 08 15:36:51.941123 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:51.941074 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d-kube-api-access-gk9sw" (OuterVolumeSpecName: "kube-api-access-gk9sw") pod "e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" (UID: "e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d"). InnerVolumeSpecName "kube-api-access-gk9sw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:36:52.040568 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:52.040477 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk9sw\" (UniqueName: \"kubernetes.io/projected/e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d-kube-api-access-gk9sw\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:36:52.626962 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:52.626921 2578 generic.go:356] "Generic (PLEG): container finished" podID="e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" containerID="71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74" exitCode=137 Sep 08 15:36:52.626962 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:52.626964 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:52.627478 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:52.627018 2578 scope.go:117] "RemoveContainer" containerID="71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74" Sep 08 15:36:52.630002 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:52.629972 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-81ae5d\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:52.638379 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:52.638249 2578 scope.go:117] "RemoveContainer" containerID="71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74" Sep 08 15:36:52.638689 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:52.638665 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74\": container with ID starting with 71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74 not found: ID does not exist" containerID="71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74" Sep 08 15:36:52.638761 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:52.638699 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74"} err="failed to get container status \"71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74\": rpc error: code = NotFound desc = could not find container \"71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74\": container with ID starting with 71db4f3632ecd35c7b86794515bc488e24feb7e720d5f79a69f65683a07d9d74 not found: ID does not exist" Sep 08 15:36:52.638899 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:36:52.638873 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-81ae5d\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" pod="opendatahub/test-curl-7749-81ae5d" Sep 08 15:36:53.502044 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:36:53.502006 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d" path="/var/lib/kubelet/pods/e1ab9a69-42a2-4da3-8f0e-ef0d6f3b8e9d/volumes" Sep 08 15:37:15.799915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:15.799824 2578 generic.go:356] "Generic (PLEG): container finished" podID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerID="cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14" exitCode=0 Sep 08 15:37:15.799915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:15.799889 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" event={"ID":"0dbb7c62-b69a-43b3-a697-bcc8338b676b","Type":"ContainerDied","Data":"cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14"} Sep 08 15:37:15.800396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:15.800324 2578 scope.go:117] "RemoveContainer" containerID="cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14" Sep 08 15:37:15.801501 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:15.801469 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:37:15.801615 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:15.801581 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:37:15.801689 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:15.801630 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:37:16.808516 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:16.808476 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" event={"ID":"0dbb7c62-b69a-43b3-a697-bcc8338b676b","Type":"ContainerStarted","Data":"b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c"} Sep 08 15:37:17.483416 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:17.483372 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:37:17.484672 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:17.484644 2578 patch_prober.go:28] interesting pod/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body= Sep 08 15:37:17.484803 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:17.484693 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" Sep 08 15:37:18.484430 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:18.484385 2578 patch_prober.go:28] interesting pod/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body= Sep 08 15:37:18.484840 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:18.484446 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" Sep 08 15:37:19.483810 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:19.483772 2578 patch_prober.go:28] interesting pod/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body= Sep 08 15:37:19.484013 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:19.483832 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" Sep 08 15:37:20.484226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:20.484176 2578 patch_prober.go:28] interesting pod/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body= Sep 08 15:37:20.484661 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:20.484257 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" Sep 08 15:37:21.484223 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:21.484187 2578 patch_prober.go:28] interesting pod/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body= Sep 08 15:37:21.484390 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:21.484244 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" Sep 08 15:37:21.619759 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:21.619714 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:37:22.487826 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:22.487793 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:37:22.488796 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:22.488777 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:37:41.626070 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:41.626034 2578 scope.go:117] "RemoveContainer" containerID="f7ec2f3463fe308016f0918b1e872f302916de770bc852ef7cf91d9f2c69f7d8" Sep 08 15:37:41.635719 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:41.635695 2578 scope.go:117] "RemoveContainer" containerID="4f7af3df88424e2b807eaff7b663af1d15cab4866ca886e6e1f42b5351b21e53" Sep 08 15:37:41.647846 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:37:41.647827 2578 scope.go:117] "RemoveContainer" containerID="ea7418a5bbcf58eda5e68d3a483f45a8c89c0e0930ab73f0c3716a350645a392" Sep 08 15:38:12.259765 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.259730 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7"] Sep 08 15:38:12.263451 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.263432 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" Sep 08 15:38:12.266690 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.266662 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:38:12.266842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.266765 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:38:12.275313 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.275285 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7"] Sep 08 15:38:12.397984 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.397942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/302470bb-7f61-42ae-81bd-461a31fdcd83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4e499427-8vld7\" (UID: \"302470bb-7f61-42ae-81bd-461a31fdcd83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" Sep 08 15:38:12.498801 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.498753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/302470bb-7f61-42ae-81bd-461a31fdcd83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4e499427-8vld7\" (UID: \"302470bb-7f61-42ae-81bd-461a31fdcd83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" Sep 08 15:38:12.499442 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.499421 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/302470bb-7f61-42ae-81bd-461a31fdcd83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4e499427-8vld7\" (UID: \"302470bb-7f61-42ae-81bd-461a31fdcd83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" Sep 08 15:38:12.575253 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.575165 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" Sep 08 15:38:12.862205 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:12.862172 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7"] Sep 08 15:38:12.863232 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:38:12.863205 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod302470bb_7f61_42ae_81bd_461a31fdcd83.slice/crio-bafe7cdf5d7ad7271a144dffc17500a465d30ef5c83fc5994fe6b86490e3cc0b WatchSource:0}: Error finding container bafe7cdf5d7ad7271a144dffc17500a465d30ef5c83fc5994fe6b86490e3cc0b: Status 404 returned error can't find the container with id bafe7cdf5d7ad7271a144dffc17500a465d30ef5c83fc5994fe6b86490e3cc0b Sep 08 15:38:13.189387 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:13.189357 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" event={"ID":"302470bb-7f61-42ae-81bd-461a31fdcd83","Type":"ContainerStarted","Data":"7c56b29a5f87ed3fde3c3a22745df4430f25b05f05f20c08ff0772480b62a18b"} Sep 08 15:38:13.189566 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:13.189406 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" event={"ID":"302470bb-7f61-42ae-81bd-461a31fdcd83","Type":"ContainerStarted","Data":"bafe7cdf5d7ad7271a144dffc17500a465d30ef5c83fc5994fe6b86490e3cc0b"} Sep 08 15:38:13.231206 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:13.230962 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" podStartSLOduration=1.2309404019999999 podStartE2EDuration="1.230940402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:38:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:38:13.230877348 +0000 UTC m=+1232.254501806" watchObservedRunningTime="2026-09-08 15:38:13.230940402 +0000 UTC m=+1232.254564858" Sep 08 15:38:14.198351 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:14.198317 2578 generic.go:356] "Generic (PLEG): container finished" podID="302470bb-7f61-42ae-81bd-461a31fdcd83" containerID="7c56b29a5f87ed3fde3c3a22745df4430f25b05f05f20c08ff0772480b62a18b" exitCode=0 Sep 08 15:38:14.198798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:14.198409 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" event={"ID":"302470bb-7f61-42ae-81bd-461a31fdcd83","Type":"ContainerDied","Data":"7c56b29a5f87ed3fde3c3a22745df4430f25b05f05f20c08ff0772480b62a18b"} Sep 08 15:38:15.351520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:15.351486 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" Sep 08 15:38:15.430930 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:15.430889 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/302470bb-7f61-42ae-81bd-461a31fdcd83-maas-api-service-ca\") pod \"302470bb-7f61-42ae-81bd-461a31fdcd83\" (UID: \"302470bb-7f61-42ae-81bd-461a31fdcd83\") " Sep 08 15:38:15.431351 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:15.431326 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/302470bb-7f61-42ae-81bd-461a31fdcd83-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "302470bb-7f61-42ae-81bd-461a31fdcd83" (UID: "302470bb-7f61-42ae-81bd-461a31fdcd83"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:38:15.532084 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:15.532048 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/302470bb-7f61-42ae-81bd-461a31fdcd83-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:38:16.212740 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:16.212710 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" Sep 08 15:38:16.212924 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:16.212742 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7" event={"ID":"302470bb-7f61-42ae-81bd-461a31fdcd83","Type":"ContainerDied","Data":"bafe7cdf5d7ad7271a144dffc17500a465d30ef5c83fc5994fe6b86490e3cc0b"} Sep 08 15:38:16.212924 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:16.212793 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bafe7cdf5d7ad7271a144dffc17500a465d30ef5c83fc5994fe6b86490e3cc0b" Sep 08 15:38:28.257879 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:28.257841 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7"] Sep 08 15:38:28.265476 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:28.265444 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4e499427-8vld7"] Sep 08 15:38:29.502210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:29.502172 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="302470bb-7f61-42ae-81bd-461a31fdcd83" path="/var/lib/kubelet/pods/302470bb-7f61-42ae-81bd-461a31fdcd83/volumes" Sep 08 15:38:41.697824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:38:41.697745 2578 scope.go:117] "RemoveContainer" containerID="eb663b3797f3aff575d25aeac77efba15f6c0c0d33f02bc0e2510e8e642f07c0" Sep 08 15:39:13.633965 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.630543 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7752-93a1b0"] Sep 08 15:39:13.633965 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.631586 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="302470bb-7f61-42ae-81bd-461a31fdcd83" containerName="revoke-keys" Sep 08 15:39:13.633965 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.631609 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="302470bb-7f61-42ae-81bd-461a31fdcd83" containerName="revoke-keys" Sep 08 15:39:13.633965 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.631858 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="302470bb-7f61-42ae-81bd-461a31fdcd83" containerName="revoke-keys" Sep 08 15:39:13.636307 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.636279 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-93a1b0" Sep 08 15:39:13.639249 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.639220 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-md8bz\"" Sep 08 15:39:13.641650 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.641622 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-93a1b0"] Sep 08 15:39:13.738817 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.738770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpkhl\" (UniqueName: \"kubernetes.io/projected/a5112bc2-2a43-4fa7-bfed-338242e02ab7-kube-api-access-jpkhl\") pod \"test-curl-7752-93a1b0\" (UID: \"a5112bc2-2a43-4fa7-bfed-338242e02ab7\") " pod="opendatahub/test-curl-7752-93a1b0" Sep 08 15:39:13.839882 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.839852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkhl\" (UniqueName: \"kubernetes.io/projected/a5112bc2-2a43-4fa7-bfed-338242e02ab7-kube-api-access-jpkhl\") pod \"test-curl-7752-93a1b0\" (UID: \"a5112bc2-2a43-4fa7-bfed-338242e02ab7\") " pod="opendatahub/test-curl-7752-93a1b0" Sep 08 15:39:13.849138 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.849076 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpkhl\" (UniqueName: \"kubernetes.io/projected/a5112bc2-2a43-4fa7-bfed-338242e02ab7-kube-api-access-jpkhl\") pod \"test-curl-7752-93a1b0\" (UID: \"a5112bc2-2a43-4fa7-bfed-338242e02ab7\") " pod="opendatahub/test-curl-7752-93a1b0" Sep 08 15:39:13.946665 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:13.946573 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-93a1b0" Sep 08 15:39:14.225499 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:14.225464 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-93a1b0"] Sep 08 15:39:14.226205 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:39:14.226172 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5112bc2_2a43_4fa7_bfed_338242e02ab7.slice/crio-93a35b974bd1be6e2c2681f68b6cefd87d8b7f96bd73c24895e1d7cf47761d68 WatchSource:0}: Error finding container 93a35b974bd1be6e2c2681f68b6cefd87d8b7f96bd73c24895e1d7cf47761d68: Status 404 returned error can't find the container with id 93a35b974bd1be6e2c2681f68b6cefd87d8b7f96bd73c24895e1d7cf47761d68 Sep 08 15:39:14.594796 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:14.594685 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-93a1b0" event={"ID":"a5112bc2-2a43-4fa7-bfed-338242e02ab7","Type":"ContainerStarted","Data":"37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c"} Sep 08 15:39:14.594796 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:14.594740 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-93a1b0" event={"ID":"a5112bc2-2a43-4fa7-bfed-338242e02ab7","Type":"ContainerStarted","Data":"93a35b974bd1be6e2c2681f68b6cefd87d8b7f96bd73c24895e1d7cf47761d68"} Sep 08 15:39:14.613466 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:14.613408 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7752-93a1b0" podStartSLOduration=1.613390319 podStartE2EDuration="1.613390319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:39:14.6105434 +0000 UTC m=+1293.634167858" watchObservedRunningTime="2026-09-08 15:39:14.613390319 +0000 UTC m=+1293.637014774" Sep 08 15:39:15.252870 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:15.252805 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7752-93a1b0"] Sep 08 15:39:15.257130 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:15.257081 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7752-93a1b0"] Sep 08 15:39:15.601335 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:39:15.601305 2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7752-93a1b0" container="test-curl-7752-93a1b0" Sep 08 15:39:16.607986 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:16.607918 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7752-93a1b0" podUID="a5112bc2-2a43-4fa7-bfed-338242e02ab7" containerName="test-curl-7752-93a1b0" containerID="cri-o://37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c" gracePeriod=2 Sep 08 15:39:18.800963 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:18.800933 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-93a1b0" Sep 08 15:39:18.893218 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:18.893106 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpkhl\" (UniqueName: \"kubernetes.io/projected/a5112bc2-2a43-4fa7-bfed-338242e02ab7-kube-api-access-jpkhl\") pod \"a5112bc2-2a43-4fa7-bfed-338242e02ab7\" (UID: \"a5112bc2-2a43-4fa7-bfed-338242e02ab7\") " Sep 08 15:39:18.895347 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:18.895321 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5112bc2-2a43-4fa7-bfed-338242e02ab7-kube-api-access-jpkhl" (OuterVolumeSpecName: "kube-api-access-jpkhl") pod "a5112bc2-2a43-4fa7-bfed-338242e02ab7" (UID: "a5112bc2-2a43-4fa7-bfed-338242e02ab7"). InnerVolumeSpecName "kube-api-access-jpkhl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:39:18.994720 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:18.994677 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpkhl\" (UniqueName: \"kubernetes.io/projected/a5112bc2-2a43-4fa7-bfed-338242e02ab7-kube-api-access-jpkhl\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:19.502241 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:19.502207 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5112bc2-2a43-4fa7-bfed-338242e02ab7" path="/var/lib/kubelet/pods/a5112bc2-2a43-4fa7-bfed-338242e02ab7/volumes" Sep 08 15:39:19.631204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:19.631151 2578 generic.go:356] "Generic (PLEG): container finished" podID="a5112bc2-2a43-4fa7-bfed-338242e02ab7" containerID="37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c" exitCode=137 Sep 08 15:39:19.631402 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:19.631276 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-93a1b0" Sep 08 15:39:19.631498 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:19.631328 2578 scope.go:117] "RemoveContainer" containerID="37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c" Sep 08 15:39:19.640661 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:19.640641 2578 scope.go:117] "RemoveContainer" containerID="37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c" Sep 08 15:39:19.640959 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:39:19.640941 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c\": container with ID starting with 37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c not found: ID does not exist" containerID="37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c" Sep 08 15:39:19.641024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:19.640970 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c"} err="failed to get container status \"37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c\": rpc error: code = NotFound desc = could not find container \"37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c\": container with ID starting with 37d0af97f49f33f64406ace5b0dd9184f82cd838aae1f4104ccc66351caf901c not found: ID does not exist" Sep 08 15:39:19.735450 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:19.735404 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g"] Sep 08 15:39:19.735683 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:19.735626 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" containerID="cri-o://b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c" gracePeriod=30 Sep 08 15:39:22.490573 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:22.490533 2578 patch_prober.go:28] interesting pod/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 15:39:22.490970 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:22.490609 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 15:39:25.000885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.000821 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:39:25.156167 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156129 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-socket\") pod \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " Sep 08 15:39:25.156357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156180 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istiod-ca-cert\") pod \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " Sep 08 15:39:25.156357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156252 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-token\") pod \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " Sep 08 15:39:25.156357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156291 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-certs\") pod \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " Sep 08 15:39:25.156357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156316 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-credential-socket\") pod \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " Sep 08 15:39:25.156357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156354 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wct5r\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-kube-api-access-wct5r\") pod \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " Sep 08 15:39:25.156648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156395 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0dbb7c62-b69a-43b3-a697-bcc8338b676b" (UID: "0dbb7c62-b69a-43b3-a697-bcc8338b676b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:39:25.156648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156456 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-envoy\") pod \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " Sep 08 15:39:25.156648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156483 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-podinfo\") pod \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " Sep 08 15:39:25.156648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156522 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-data\") pod \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\" (UID: \"0dbb7c62-b69a-43b3-a697-bcc8338b676b\") " Sep 08 15:39:25.156865 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156659 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0dbb7c62-b69a-43b3-a697-bcc8338b676b" (UID: "0dbb7c62-b69a-43b3-a697-bcc8338b676b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:39:25.156865 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156674 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0dbb7c62-b69a-43b3-a697-bcc8338b676b" (UID: "0dbb7c62-b69a-43b3-a697-bcc8338b676b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:39:25.156865 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156737 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0dbb7c62-b69a-43b3-a697-bcc8338b676b" (UID: "0dbb7c62-b69a-43b3-a697-bcc8338b676b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:39:25.157032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156883 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-data" (OuterVolumeSpecName: "istio-data") pod "0dbb7c62-b69a-43b3-a697-bcc8338b676b" (UID: "0dbb7c62-b69a-43b3-a697-bcc8338b676b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:39:25.157032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156971 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:25.157032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.156989 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:25.157032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.157003 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:25.157032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.157016 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:25.157032 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.157028 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:25.158708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.158680 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-token" (OuterVolumeSpecName: "istio-token") pod "0dbb7c62-b69a-43b3-a697-bcc8338b676b" (UID: "0dbb7c62-b69a-43b3-a697-bcc8338b676b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:39:25.159056 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.159040 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0dbb7c62-b69a-43b3-a697-bcc8338b676b" (UID: "0dbb7c62-b69a-43b3-a697-bcc8338b676b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:39:25.159239 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.159213 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0dbb7c62-b69a-43b3-a697-bcc8338b676b" (UID: "0dbb7c62-b69a-43b3-a697-bcc8338b676b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:39:25.159239 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.159223 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-kube-api-access-wct5r" (OuterVolumeSpecName: "kube-api-access-wct5r") pod "0dbb7c62-b69a-43b3-a697-bcc8338b676b" (UID: "0dbb7c62-b69a-43b3-a697-bcc8338b676b"). InnerVolumeSpecName "kube-api-access-wct5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:39:25.258456 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.258362 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:25.258456 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.258394 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wct5r\" (UniqueName: \"kubernetes.io/projected/0dbb7c62-b69a-43b3-a697-bcc8338b676b-kube-api-access-wct5r\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:25.258456 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.258403 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:25.258456 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.258412 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dbb7c62-b69a-43b3-a697-bcc8338b676b-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:39:25.672009 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.671973 2578 generic.go:356] "Generic (PLEG): container finished" podID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerID="b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c" exitCode=0 Sep 08 15:39:25.672247 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.672017 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" Sep 08 15:39:25.672247 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.672055 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" event={"ID":"0dbb7c62-b69a-43b3-a697-bcc8338b676b","Type":"ContainerDied","Data":"b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c"} Sep 08 15:39:25.672247 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.672123 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g" event={"ID":"0dbb7c62-b69a-43b3-a697-bcc8338b676b","Type":"ContainerDied","Data":"18f8aa6d4d04594551254084c1984858225d2a0ad3004525c0f8460844ed3a71"} Sep 08 15:39:25.672247 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.672139 2578 scope.go:117] "RemoveContainer" containerID="b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c" Sep 08 15:39:25.682102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.682080 2578 scope.go:117] "RemoveContainer" containerID="cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14" Sep 08 15:39:25.691753 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.691730 2578 scope.go:117] "RemoveContainer" containerID="b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c" Sep 08 15:39:25.692145 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:39:25.692102 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c\": container with ID starting with b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c not found: ID does not exist" containerID="b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c" Sep 08 15:39:25.692221 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.692155 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c"} err="failed to get container status \"b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c\": rpc error: code = NotFound desc = could not find container \"b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c\": container with ID starting with b3fd35fcd7fd3feb98560335a117ac450c17489995caf9555849af69fae3da0c not found: ID does not exist" Sep 08 15:39:25.692221 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.692175 2578 scope.go:117] "RemoveContainer" containerID="cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14" Sep 08 15:39:25.692502 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:39:25.692485 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14\": container with ID starting with cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14 not found: ID does not exist" containerID="cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14" Sep 08 15:39:25.692559 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.692512 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14"} err="failed to get container status \"cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14\": rpc error: code = NotFound desc = could not find container \"cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14\": container with ID starting with cb90cd2560cd5e5f350893ab877ade75f60f74e37041a528f3fe963229701c14 not found: ID does not exist" Sep 08 15:39:25.696337 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.696308 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g"] Sep 08 15:39:25.701533 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:25.701502 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4e499427-openshift-default-68475d96bd-jxk9g"] Sep 08 15:39:27.502183 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:27.502147 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" path="/var/lib/kubelet/pods/0dbb7c62-b69a-43b3-a697-bcc8338b676b/volumes" Sep 08 15:39:41.723316 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:41.723281 2578 scope.go:117] "RemoveContainer" containerID="9d2b8eb080ba10a9166d825ec19f3cc2a1ebc79b9fc089470bd7ef2adf1dc9b2" Sep 08 15:39:41.732634 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:39:41.732611 2578 scope.go:117] "RemoveContainer" containerID="0e46c9bed1a19e8526d59244afca36f1c3da5811d89a390aa91adf4cebacc491" Sep 08 15:40:14.971273 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.971177 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt"] Sep 08 15:40:14.971919 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.971898 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" Sep 08 15:40:14.971969 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.971923 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" Sep 08 15:40:14.971969 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.971934 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" Sep 08 15:40:14.971969 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.971942 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" Sep 08 15:40:14.972064 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.971972 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5112bc2-2a43-4fa7-bfed-338242e02ab7" containerName="test-curl-7752-93a1b0" Sep 08 15:40:14.972064 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.971982 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5112bc2-2a43-4fa7-bfed-338242e02ab7" containerName="test-curl-7752-93a1b0" Sep 08 15:40:14.972153 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.972143 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" Sep 08 15:40:14.972193 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.972158 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbb7c62-b69a-43b3-a697-bcc8338b676b" containerName="istio-proxy" Sep 08 15:40:14.972193 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.972169 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5112bc2-2a43-4fa7-bfed-338242e02ab7" containerName="test-curl-7752-93a1b0" Sep 08 15:40:14.975908 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.975884 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:14.979179 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.979154 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-630e4d-openshift-default-dockercfg-w6ncz\"" Sep 08 15:40:14.992939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:14.992903 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt"] Sep 08 15:40:15.071246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.071202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-socket\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.071446 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.071338 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-certs\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.071446 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.071366 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-data\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.071446 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.071391 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-istio-token\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.071446 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.071420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfbwp\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-kube-api-access-vfbwp\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.071682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.071462 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-credential-socket\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.071682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.071483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-envoy\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.071682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.071536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86365d36-8a71-4893-b920-30502e119451-istiod-ca-cert\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.071682 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.071583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86365d36-8a71-4893-b920-30502e119451-istio-podinfo\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172259 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-socket\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172454 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172306 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-certs\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172454 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-data\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172454 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172348 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-istio-token\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172454 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfbwp\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-kube-api-access-vfbwp\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172454 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-credential-socket\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172454 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172408 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-envoy\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86365d36-8a71-4893-b920-30502e119451-istiod-ca-cert\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172681 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-socket\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172690 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86365d36-8a71-4893-b920-30502e119451-istio-podinfo\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172755 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-data\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172757 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-certs\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.172839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.172811 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-credential-socket\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.173233 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.173214 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86365d36-8a71-4893-b920-30502e119451-istiod-ca-cert\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.174884 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.174862 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-envoy\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.175191 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.175171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86365d36-8a71-4893-b920-30502e119451-istio-podinfo\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.183568 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.183536 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfbwp\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-kube-api-access-vfbwp\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.184501 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.184481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-istio-token\") pod \"e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.286829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.286732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:15.574011 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.573974 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt"] Sep 08 15:40:15.574583 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:40:15.574558 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86365d36_8a71_4893_b920_30502e119451.slice/crio-389d1bde2a34bfea80499b13799e56eb6d747db4e600e21daa2c9c31e96dc0ff WatchSource:0}: Error finding container 389d1bde2a34bfea80499b13799e56eb6d747db4e600e21daa2c9c31e96dc0ff: Status 404 returned error can't find the container with id 389d1bde2a34bfea80499b13799e56eb6d747db4e600e21daa2c9c31e96dc0ff Sep 08 15:40:15.576824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.576792 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:40:15.576934 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.576862 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:40:15.576934 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.576898 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:40:15.995429 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.995390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" event={"ID":"86365d36-8a71-4893-b920-30502e119451","Type":"ContainerStarted","Data":"dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045"} Sep 08 15:40:15.995880 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:15.995446 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" event={"ID":"86365d36-8a71-4893-b920-30502e119451","Type":"ContainerStarted","Data":"389d1bde2a34bfea80499b13799e56eb6d747db4e600e21daa2c9c31e96dc0ff"} Sep 08 15:40:16.020791 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:16.020738 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" podStartSLOduration=2.02071861 podStartE2EDuration="2.02071861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:40:16.018484888 +0000 UTC m=+1355.042109327" watchObservedRunningTime="2026-09-08 15:40:16.02071861 +0000 UTC m=+1355.044343068" Sep 08 15:40:16.287247 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:16.287150 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:16.288771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:16.288744 2578 patch_prober.go:28] interesting pod/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body= Sep 08 15:40:16.288896 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:16.288794 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" Sep 08 15:40:17.291167 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:17.291137 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:18.009020 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:18.008987 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:18.010094 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:18.010077 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:40.163257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:40.163170 2578 generic.go:356] "Generic (PLEG): container finished" podID="86365d36-8a71-4893-b920-30502e119451" containerID="dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045" exitCode=0 Sep 08 15:40:40.163257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:40.163245 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" event={"ID":"86365d36-8a71-4893-b920-30502e119451","Type":"ContainerDied","Data":"dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045"} Sep 08 15:40:40.163709 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:40.163695 2578 scope.go:117] "RemoveContainer" containerID="dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045" Sep 08 15:40:40.164834 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:40.164807 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:40:40.164897 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:40.164875 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:40:40.164943 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:40.164908 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:40:41.172802 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:41.172766 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" event={"ID":"86365d36-8a71-4893-b920-30502e119451","Type":"ContainerStarted","Data":"f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe"} Sep 08 15:40:41.287941 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:41.287899 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:41.288209 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:41.288183 2578 patch_prober.go:28] interesting pod/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body= Sep 08 15:40:41.288286 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:41.288262 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" Sep 08 15:40:41.772142 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:41.772088 2578 scope.go:117] "RemoveContainer" containerID="95409ea636222f85f4eefaed0ab866145706bae4ebc460982f74969137aeab90" Sep 08 15:40:42.287585 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:42.287540 2578 patch_prober.go:28] interesting pod/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body= Sep 08 15:40:42.288060 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:42.287610 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" Sep 08 15:40:43.287297 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.287258 2578 patch_prober.go:28] interesting pod/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body= Sep 08 15:40:43.287482 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.287315 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" Sep 08 15:40:43.680940 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.680843 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c"] Sep 08 15:40:43.688622 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.688580 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.695837 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.695757 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-bea11f-openshift-default-dockercfg-7mv2s\"" Sep 08 15:40:43.701987 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.701960 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c"] Sep 08 15:40:43.764835 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.764792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-data\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.765012 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.764907 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-certs\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.765012 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.764939 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-token\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.765012 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.764962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-credential-socket\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.765012 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.764995 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/193f0014-aaec-45f4-ac12-24c3f2f5f733-istiod-ca-cert\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.765210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.765035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-envoy\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.765210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.765092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjkpn\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-kube-api-access-cjkpn\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.765210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.765170 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-podinfo\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.765210 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.765201 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-socket\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.866377 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.866331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-podinfo\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.866582 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.866404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-socket\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.866582 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.866464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-data\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.866582 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.866542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-certs\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.866947 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.866923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-token\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.867420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.867396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-credential-socket\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.867622 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.867604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/193f0014-aaec-45f4-ac12-24c3f2f5f733-istiod-ca-cert\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.867764 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.867748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-envoy\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.868267 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.868247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjkpn\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-kube-api-access-cjkpn\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.868489 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.867676 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-credential-socket\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.868688 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.866974 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-data\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.868810 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.867030 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-certs\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.868930 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.868308 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/193f0014-aaec-45f4-ac12-24c3f2f5f733-istiod-ca-cert\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.869056 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.866922 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-socket\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.873410 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.873381 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-envoy\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.873725 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.873706 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-podinfo\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.878811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.878775 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-token\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.879716 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.879670 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjkpn\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-kube-api-access-cjkpn\") pod \"e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:43.999912 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:43.999872 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:44.287845 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:44.287748 2578 patch_prober.go:28] interesting pod/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body= Sep 08 15:40:44.287845 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:44.287811 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" Sep 08 15:40:44.300360 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:44.300335 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c"] Sep 08 15:40:44.302253 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:40:44.302226 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod193f0014_aaec_45f4_ac12_24c3f2f5f733.slice/crio-5b308118273a3d399b552d958dbddad752804f36763e3855108459cb4d7c62c0 WatchSource:0}: Error finding container 5b308118273a3d399b552d958dbddad752804f36763e3855108459cb4d7c62c0: Status 404 returned error can't find the container with id 5b308118273a3d399b552d958dbddad752804f36763e3855108459cb4d7c62c0 Sep 08 15:40:44.305041 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:44.305003 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:40:44.305143 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:44.305089 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:40:44.305192 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:44.305149 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:40:45.197842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:45.197802 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" event={"ID":"193f0014-aaec-45f4-ac12-24c3f2f5f733","Type":"ContainerStarted","Data":"73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d"} Sep 08 15:40:45.198312 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:45.197860 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" event={"ID":"193f0014-aaec-45f4-ac12-24c3f2f5f733","Type":"ContainerStarted","Data":"5b308118273a3d399b552d958dbddad752804f36763e3855108459cb4d7c62c0"} Sep 08 15:40:45.223901 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:45.223849 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" podStartSLOduration=2.223832135 podStartE2EDuration="2.223832135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:40:45.221475714 +0000 UTC m=+1384.245100170" watchObservedRunningTime="2026-09-08 15:40:45.223832135 +0000 UTC m=+1384.247456618" Sep 08 15:40:45.287437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:45.287403 2578 patch_prober.go:28] interesting pod/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body= Sep 08 15:40:45.287607 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:45.287453 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" Sep 08 15:40:46.000704 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:46.000667 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:46.005540 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:46.005507 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:46.203710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:46.203679 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:46.204662 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:46.204640 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:40:46.287464 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:46.287374 2578 patch_prober.go:28] interesting pod/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body= Sep 08 15:40:46.287464 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:46.287429 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" Sep 08 15:40:47.291604 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:47.291571 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:47.291983 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:47.291836 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:40:47.292746 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:40:47.292729 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:41:08.346915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:08.346834 2578 generic.go:356] "Generic (PLEG): container finished" podID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerID="73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d" exitCode=0 Sep 08 15:41:08.347332 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:08.346912 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" event={"ID":"193f0014-aaec-45f4-ac12-24c3f2f5f733","Type":"ContainerDied","Data":"73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d"} Sep 08 15:41:08.347392 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:08.347377 2578 scope.go:117] "RemoveContainer" containerID="73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d" Sep 08 15:41:08.348658 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:08.348630 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:41:08.348724 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:08.348700 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:41:08.348768 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:08.348731 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:41:09.356900 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:09.356866 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" event={"ID":"193f0014-aaec-45f4-ac12-24c3f2f5f733","Type":"ContainerStarted","Data":"3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809"} Sep 08 15:41:10.000769 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:10.000729 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:41:10.000955 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:10.000922 2578 patch_prober.go:28] interesting pod/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 08 15:41:10.001008 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:10.000987 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 08 15:41:11.001203 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:11.001158 2578 patch_prober.go:28] interesting pod/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 08 15:41:11.001611 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:11.001234 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 08 15:41:12.001051 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:12.001011 2578 patch_prober.go:28] interesting pod/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 08 15:41:12.001247 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:12.001073 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 08 15:41:13.000845 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:13.000799 2578 patch_prober.go:28] interesting pod/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 08 15:41:13.001044 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:13.000878 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 08 15:41:14.001192 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:14.001153 2578 patch_prober.go:28] interesting pod/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 08 15:41:14.001587 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:14.001208 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 08 15:41:15.004741 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.004699 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:41:15.005143 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.004976 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:41:15.005870 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.005851 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:41:15.123369 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.123326 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx"] Sep 08 15:41:15.127714 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.127690 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" Sep 08 15:41:15.131006 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.130550 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:41:15.131006 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.130558 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:41:15.134505 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.134463 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx"] Sep 08 15:41:15.205935 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.205894 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315d02d4-a5fc-441c-a959-bf6a3cf77335-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-630e4d-stwpx\" (UID: \"315d02d4-a5fc-441c-a959-bf6a3cf77335\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" Sep 08 15:41:15.306872 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.306783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315d02d4-a5fc-441c-a959-bf6a3cf77335-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-630e4d-stwpx\" (UID: \"315d02d4-a5fc-441c-a959-bf6a3cf77335\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" Sep 08 15:41:15.307497 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.307476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315d02d4-a5fc-441c-a959-bf6a3cf77335-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-630e4d-stwpx\" (UID: \"315d02d4-a5fc-441c-a959-bf6a3cf77335\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" Sep 08 15:41:15.443224 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.443182 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" Sep 08 15:41:15.720282 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:15.720253 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx"] Sep 08 15:41:15.722060 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:41:15.722018 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod315d02d4_a5fc_441c_a959_bf6a3cf77335.slice/crio-f81039f7370e35de14ec1f20fffe216913ddcea095925cf01d80b8e6d541ee89 WatchSource:0}: Error finding container f81039f7370e35de14ec1f20fffe216913ddcea095925cf01d80b8e6d541ee89: Status 404 returned error can't find the container with id f81039f7370e35de14ec1f20fffe216913ddcea095925cf01d80b8e6d541ee89 Sep 08 15:41:16.403184 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:16.403084 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" event={"ID":"315d02d4-a5fc-441c-a959-bf6a3cf77335","Type":"ContainerStarted","Data":"90c4cdfa8cb57dd4b9417bbbf3bea5fe4cf2a473915ed81b1e57766d11dd377f"} Sep 08 15:41:16.403184 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:16.403167 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" event={"ID":"315d02d4-a5fc-441c-a959-bf6a3cf77335","Type":"ContainerStarted","Data":"f81039f7370e35de14ec1f20fffe216913ddcea095925cf01d80b8e6d541ee89"} Sep 08 15:41:16.421730 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:16.421670 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" podStartSLOduration=1.421648514 podStartE2EDuration="1.421648514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:41:16.420689828 +0000 UTC m=+1415.444314286" watchObservedRunningTime="2026-09-08 15:41:16.421648514 +0000 UTC m=+1415.445272969" Sep 08 15:41:17.411698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:17.411660 2578 generic.go:356] "Generic (PLEG): container finished" podID="315d02d4-a5fc-441c-a959-bf6a3cf77335" containerID="90c4cdfa8cb57dd4b9417bbbf3bea5fe4cf2a473915ed81b1e57766d11dd377f" exitCode=0 Sep 08 15:41:17.412093 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:17.411739 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" event={"ID":"315d02d4-a5fc-441c-a959-bf6a3cf77335","Type":"ContainerDied","Data":"90c4cdfa8cb57dd4b9417bbbf3bea5fe4cf2a473915ed81b1e57766d11dd377f"} Sep 08 15:41:18.579150 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:18.579103 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" Sep 08 15:41:18.643206 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:18.643162 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315d02d4-a5fc-441c-a959-bf6a3cf77335-maas-api-service-ca\") pod \"315d02d4-a5fc-441c-a959-bf6a3cf77335\" (UID: \"315d02d4-a5fc-441c-a959-bf6a3cf77335\") " Sep 08 15:41:18.643534 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:18.643508 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/315d02d4-a5fc-441c-a959-bf6a3cf77335-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "315d02d4-a5fc-441c-a959-bf6a3cf77335" (UID: "315d02d4-a5fc-441c-a959-bf6a3cf77335"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:41:18.744488 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:18.744446 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/315d02d4-a5fc-441c-a959-bf6a3cf77335-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:19.428952 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:19.428919 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" Sep 08 15:41:19.429176 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:19.428962 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx" event={"ID":"315d02d4-a5fc-441c-a959-bf6a3cf77335","Type":"ContainerDied","Data":"f81039f7370e35de14ec1f20fffe216913ddcea095925cf01d80b8e6d541ee89"} Sep 08 15:41:19.429176 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:19.429009 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f81039f7370e35de14ec1f20fffe216913ddcea095925cf01d80b8e6d541ee89" Sep 08 15:41:20.678287 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.678245 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7752-763781"] Sep 08 15:41:20.678728 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.678713 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="315d02d4-a5fc-441c-a959-bf6a3cf77335" containerName="revoke-keys" Sep 08 15:41:20.678773 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.678729 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="315d02d4-a5fc-441c-a959-bf6a3cf77335" containerName="revoke-keys" Sep 08 15:41:20.678835 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.678825 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="315d02d4-a5fc-441c-a959-bf6a3cf77335" containerName="revoke-keys" Sep 08 15:41:20.682087 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.682068 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:20.685592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.685565 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-md8bz\"" Sep 08 15:41:20.689373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.689342 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-763781"] Sep 08 15:41:20.766554 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.766515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5r4j\" (UniqueName: \"kubernetes.io/projected/ebf331c5-2daf-448a-99ae-5f551b263a42-kube-api-access-n5r4j\") pod \"test-curl-7752-763781\" (UID: \"ebf331c5-2daf-448a-99ae-5f551b263a42\") " pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:20.867359 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.867317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5r4j\" (UniqueName: \"kubernetes.io/projected/ebf331c5-2daf-448a-99ae-5f551b263a42-kube-api-access-n5r4j\") pod \"test-curl-7752-763781\" (UID: \"ebf331c5-2daf-448a-99ae-5f551b263a42\") " pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:20.880663 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.880635 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5r4j\" (UniqueName: \"kubernetes.io/projected/ebf331c5-2daf-448a-99ae-5f551b263a42-kube-api-access-n5r4j\") pod \"test-curl-7752-763781\" (UID: \"ebf331c5-2daf-448a-99ae-5f551b263a42\") " pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:20.993518 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:20.993481 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:21.268684 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:21.268655 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-763781"] Sep 08 15:41:21.270036 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:41:21.270007 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebf331c5_2daf_448a_99ae_5f551b263a42.slice/crio-0f5eb231b06987aef42206ee2a7b34d8994ddebc3a8dbe5bc59e8692b3d506ab WatchSource:0}: Error finding container 0f5eb231b06987aef42206ee2a7b34d8994ddebc3a8dbe5bc59e8692b3d506ab: Status 404 returned error can't find the container with id 0f5eb231b06987aef42206ee2a7b34d8994ddebc3a8dbe5bc59e8692b3d506ab Sep 08 15:41:21.446951 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:21.446909 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-763781" event={"ID":"ebf331c5-2daf-448a-99ae-5f551b263a42","Type":"ContainerStarted","Data":"e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5"} Sep 08 15:41:21.447160 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:21.446964 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-763781" event={"ID":"ebf331c5-2daf-448a-99ae-5f551b263a42","Type":"ContainerStarted","Data":"0f5eb231b06987aef42206ee2a7b34d8994ddebc3a8dbe5bc59e8692b3d506ab"} Sep 08 15:41:21.466820 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:21.466761 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7752-763781" podStartSLOduration=1.466744562 podStartE2EDuration="1.466744562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:41:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:41:21.464862117 +0000 UTC m=+1420.488486574" watchObservedRunningTime="2026-09-08 15:41:21.466744562 +0000 UTC m=+1420.490369018" Sep 08 15:41:22.067405 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.067368 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7752-763781"] Sep 08 15:41:22.071362 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.071319 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7752-763781"] Sep 08 15:41:22.180062 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.180027 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7752-1eaa9b"] Sep 08 15:41:22.180492 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.180480 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf331c5-2daf-448a-99ae-5f551b263a42" containerName="test-curl-7752-763781" Sep 08 15:41:22.180554 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.180494 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf331c5-2daf-448a-99ae-5f551b263a42" containerName="test-curl-7752-763781" Sep 08 15:41:22.180591 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.180587 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebf331c5-2daf-448a-99ae-5f551b263a42" containerName="test-curl-7752-763781" Sep 08 15:41:22.183918 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.183902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-1eaa9b" Sep 08 15:41:22.192849 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.192822 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-1eaa9b"] Sep 08 15:41:22.284074 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.284030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rlnk\" (UniqueName: \"kubernetes.io/projected/c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528-kube-api-access-7rlnk\") pod \"test-curl-7752-1eaa9b\" (UID: \"c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528\") " pod="opendatahub/test-curl-7752-1eaa9b" Sep 08 15:41:22.384929 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.384822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rlnk\" (UniqueName: \"kubernetes.io/projected/c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528-kube-api-access-7rlnk\") pod \"test-curl-7752-1eaa9b\" (UID: \"c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528\") " pod="opendatahub/test-curl-7752-1eaa9b" Sep 08 15:41:22.395471 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.395430 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rlnk\" (UniqueName: \"kubernetes.io/projected/c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528-kube-api-access-7rlnk\") pod \"test-curl-7752-1eaa9b\" (UID: \"c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528\") " pod="opendatahub/test-curl-7752-1eaa9b" Sep 08 15:41:22.453779 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:22.453748 2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7752-763781" container="test-curl-7752-763781" Sep 08 15:41:22.495346 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.495315 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-1eaa9b" Sep 08 15:41:22.770738 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:22.770715 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-1eaa9b"] Sep 08 15:41:22.772443 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:41:22.772406 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3d0d7d3_17d6_4a5e_a0c9_99c7715c3528.slice/crio-e56cec5b8dec8d03e150a4d3e6f120870b2d0663a5ae4e0030a99738e1fa5fb0 WatchSource:0}: Error finding container e56cec5b8dec8d03e150a4d3e6f120870b2d0663a5ae4e0030a99738e1fa5fb0: Status 404 returned error can't find the container with id e56cec5b8dec8d03e150a4d3e6f120870b2d0663a5ae4e0030a99738e1fa5fb0 Sep 08 15:41:23.462010 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:23.461971 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-1eaa9b" event={"ID":"c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528","Type":"ContainerStarted","Data":"189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4"} Sep 08 15:41:23.462520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:23.462019 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-1eaa9b" event={"ID":"c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528","Type":"ContainerStarted","Data":"e56cec5b8dec8d03e150a4d3e6f120870b2d0663a5ae4e0030a99738e1fa5fb0"} Sep 08 15:41:23.462520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:23.462153 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7752-763781" podUID="ebf331c5-2daf-448a-99ae-5f551b263a42" containerName="test-curl-7752-763781" containerID="cri-o://e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5" gracePeriod=2 Sep 08 15:41:23.464727 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:23.464700 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-763781\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="ebf331c5-2daf-448a-99ae-5f551b263a42" pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:23.481408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:23.481356 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7752-1eaa9b" podStartSLOduration=1.481335317 podStartE2EDuration="1.481335317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:41:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:41:23.47939626 +0000 UTC m=+1422.503020718" watchObservedRunningTime="2026-09-08 15:41:23.481335317 +0000 UTC m=+1422.504959776" Sep 08 15:41:23.481565 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:23.481472 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-763781\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="ebf331c5-2daf-448a-99ae-5f551b263a42" pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:24.071009 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:24.070972 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7752-1eaa9b"] Sep 08 15:41:24.075436 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:24.075399 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7752-1eaa9b"] Sep 08 15:41:24.468078 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:24.468043 2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7752-1eaa9b" container="test-curl-7752-1eaa9b" Sep 08 15:41:25.475501 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:25.475460 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7752-1eaa9b" podUID="c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528" containerName="test-curl-7752-1eaa9b" containerID="cri-o://189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4" gracePeriod=2 Sep 08 15:41:25.477944 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:25.477903 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-1eaa9b\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528" pod="opendatahub/test-curl-7752-1eaa9b" Sep 08 15:41:25.621688 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:25.621656 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:25.722040 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:25.721937 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5r4j\" (UniqueName: \"kubernetes.io/projected/ebf331c5-2daf-448a-99ae-5f551b263a42-kube-api-access-n5r4j\") pod \"ebf331c5-2daf-448a-99ae-5f551b263a42\" (UID: \"ebf331c5-2daf-448a-99ae-5f551b263a42\") " Sep 08 15:41:25.724404 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:25.724373 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf331c5-2daf-448a-99ae-5f551b263a42-kube-api-access-n5r4j" (OuterVolumeSpecName: "kube-api-access-n5r4j") pod "ebf331c5-2daf-448a-99ae-5f551b263a42" (UID: "ebf331c5-2daf-448a-99ae-5f551b263a42"). InnerVolumeSpecName "kube-api-access-n5r4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:41:25.823325 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:25.823281 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5r4j\" (UniqueName: \"kubernetes.io/projected/ebf331c5-2daf-448a-99ae-5f551b263a42-kube-api-access-n5r4j\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:26.483100 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:26.483062 2578 generic.go:356] "Generic (PLEG): container finished" podID="ebf331c5-2daf-448a-99ae-5f551b263a42" containerID="e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5" exitCode=137 Sep 08 15:41:26.483600 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:26.483127 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:26.483600 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:26.483167 2578 scope.go:117] "RemoveContainer" containerID="e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5" Sep 08 15:41:26.492833 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:26.492813 2578 scope.go:117] "RemoveContainer" containerID="e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5" Sep 08 15:41:26.493174 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:26.493155 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5\": container with ID starting with e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5 not found: ID does not exist" containerID="e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5" Sep 08 15:41:26.493227 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:26.493184 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5"} err="failed to get container status \"e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5\": rpc error: code = NotFound desc = could not find container \"e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5\": container with ID starting with e12e1c9e742ff823154ed92c27f872aa4602e8866aba72f6b09500002b3c91b5 not found: ID does not exist" Sep 08 15:41:26.493802 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:26.493784 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-763781\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="ebf331c5-2daf-448a-99ae-5f551b263a42" pod="opendatahub/test-curl-7752-763781" Sep 08 15:41:27.502859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:27.502824 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebf331c5-2daf-448a-99ae-5f551b263a42" path="/var/lib/kubelet/pods/ebf331c5-2daf-448a-99ae-5f551b263a42/volumes" Sep 08 15:41:27.661024 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:27.661000 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-1eaa9b" Sep 08 15:41:27.843064 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:27.842978 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rlnk\" (UniqueName: \"kubernetes.io/projected/c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528-kube-api-access-7rlnk\") pod \"c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528\" (UID: \"c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528\") " Sep 08 15:41:27.845288 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:27.845257 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528-kube-api-access-7rlnk" (OuterVolumeSpecName: "kube-api-access-7rlnk") pod "c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528" (UID: "c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528"). InnerVolumeSpecName "kube-api-access-7rlnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:41:27.944471 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:27.944433 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rlnk\" (UniqueName: \"kubernetes.io/projected/c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528-kube-api-access-7rlnk\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:28.499959 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:28.499924 2578 generic.go:356] "Generic (PLEG): container finished" podID="c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528" containerID="189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4" exitCode=137 Sep 08 15:41:28.500194 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:28.499969 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-1eaa9b" Sep 08 15:41:28.500194 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:28.500013 2578 scope.go:117] "RemoveContainer" containerID="189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4" Sep 08 15:41:28.509829 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:28.509809 2578 scope.go:117] "RemoveContainer" containerID="189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4" Sep 08 15:41:28.510186 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:28.510159 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4\": container with ID starting with 189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4 not found: ID does not exist" containerID="189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4" Sep 08 15:41:28.510277 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:28.510189 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4"} err="failed to get container status \"189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4\": rpc error: code = NotFound desc = could not find container \"189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4\": container with ID starting with 189d1dd6fa74e6e1be9688299a87738614d8e1659956353ca4ad470218d6d6b4 not found: ID does not exist" Sep 08 15:41:28.510456 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:28.510435 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-1eaa9b\" is forbidden: User \"system:node:ip-10-0-132-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-25.ec2.internal' and this object" podUID="c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528" pod="opendatahub/test-curl-7752-1eaa9b" Sep 08 15:41:29.503669 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:29.503633 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528" path="/var/lib/kubelet/pods/c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528/volumes" Sep 08 15:41:31.118572 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:31.118525 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx"] Sep 08 15:41:31.124078 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:31.124040 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-630e4d-stwpx"] Sep 08 15:41:31.502781 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:31.502748 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="315d02d4-a5fc-441c-a959-bf6a3cf77335" path="/var/lib/kubelet/pods/315d02d4-a5fc-441c-a959-bf6a3cf77335/volumes" Sep 08 15:41:37.923406 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:37.923321 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt"] Sep 08 15:41:37.923792 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:37.923534 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" containerID="cri-o://f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe" gracePeriod=30 Sep 08 15:41:38.259366 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.259320 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv"] Sep 08 15:41:38.259855 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.259836 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528" containerName="test-curl-7752-1eaa9b" Sep 08 15:41:38.259937 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.259858 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528" containerName="test-curl-7752-1eaa9b" Sep 08 15:41:38.259999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.259991 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d0d7d3-17d6-4a5e-a0c9-99c7715c3528" containerName="test-curl-7752-1eaa9b" Sep 08 15:41:38.264646 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.264623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" Sep 08 15:41:38.267408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.267384 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:41:38.267501 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.267482 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:41:38.272691 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.272663 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv"] Sep 08 15:41:38.354386 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.354344 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d4be4d-12de-4010-8478-217518b0a17f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv\" (UID: \"86d4be4d-12de-4010-8478-217518b0a17f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" Sep 08 15:41:38.455023 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.454980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d4be4d-12de-4010-8478-217518b0a17f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv\" (UID: \"86d4be4d-12de-4010-8478-217518b0a17f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" Sep 08 15:41:38.455621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.455601 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d4be4d-12de-4010-8478-217518b0a17f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv\" (UID: \"86d4be4d-12de-4010-8478-217518b0a17f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" Sep 08 15:41:38.578321 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.578229 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" Sep 08 15:41:38.867912 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:38.867880 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv"] Sep 08 15:41:38.869312 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:41:38.869270 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d4be4d_12de_4010_8478_217518b0a17f.slice/crio-8ceed75523e86f4798f3124e6342a424788ce33273fb65488c7ba6a5edad56d1 WatchSource:0}: Error finding container 8ceed75523e86f4798f3124e6342a424788ce33273fb65488c7ba6a5edad56d1: Status 404 returned error can't find the container with id 8ceed75523e86f4798f3124e6342a424788ce33273fb65488c7ba6a5edad56d1 Sep 08 15:41:39.574512 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:39.574465 2578 generic.go:356] "Generic (PLEG): container finished" podID="86d4be4d-12de-4010-8478-217518b0a17f" containerID="0eba9643df6683e159223776555af5ca506d46bd5527507b647e97a01ec3bf85" exitCode=0 Sep 08 15:41:39.574909 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:39.574544 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" event={"ID":"86d4be4d-12de-4010-8478-217518b0a17f","Type":"ContainerDied","Data":"0eba9643df6683e159223776555af5ca506d46bd5527507b647e97a01ec3bf85"} Sep 08 15:41:39.574909 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:39.574595 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" event={"ID":"86d4be4d-12de-4010-8478-217518b0a17f","Type":"ContainerStarted","Data":"8ceed75523e86f4798f3124e6342a424788ce33273fb65488c7ba6a5edad56d1"} Sep 08 15:41:40.730922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:40.730897 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" Sep 08 15:41:40.781330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:40.781294 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d4be4d-12de-4010-8478-217518b0a17f-maas-api-service-ca\") pod \"86d4be4d-12de-4010-8478-217518b0a17f\" (UID: \"86d4be4d-12de-4010-8478-217518b0a17f\") " Sep 08 15:41:40.781665 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:40.781643 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86d4be4d-12de-4010-8478-217518b0a17f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "86d4be4d-12de-4010-8478-217518b0a17f" (UID: "86d4be4d-12de-4010-8478-217518b0a17f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:41:40.883342 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:40.883251 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d4be4d-12de-4010-8478-217518b0a17f-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:41.589557 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:41.589525 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" Sep 08 15:41:41.589744 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:41.589559 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv" event={"ID":"86d4be4d-12de-4010-8478-217518b0a17f","Type":"ContainerDied","Data":"8ceed75523e86f4798f3124e6342a424788ce33273fb65488c7ba6a5edad56d1"} Sep 08 15:41:41.589744 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:41.589604 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ceed75523e86f4798f3124e6342a424788ce33273fb65488c7ba6a5edad56d1" Sep 08 15:41:41.797458 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:41.797368 2578 scope.go:117] "RemoveContainer" containerID="2afc8471ffb8777e12e0ff4c28610afb109499c3fe437ea8702a18e401fcb4a0" Sep 08 15:41:41.807584 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:41.807560 2578 scope.go:117] "RemoveContainer" containerID="c17d72b8cd8c3e4c2043b46ddc7515a34e7e739ea4c34855ddf4c96323ffd386" Sep 08 15:41:43.189309 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.189235 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:41:43.308770 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.308736 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-data\") pod \"86365d36-8a71-4893-b920-30502e119451\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " Sep 08 15:41:43.308965 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.308780 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfbwp\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-kube-api-access-vfbwp\") pod \"86365d36-8a71-4893-b920-30502e119451\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " Sep 08 15:41:43.308965 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.308839 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86365d36-8a71-4893-b920-30502e119451-istiod-ca-cert\") pod \"86365d36-8a71-4893-b920-30502e119451\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " Sep 08 15:41:43.308965 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.308879 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-certs\") pod \"86365d36-8a71-4893-b920-30502e119451\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " Sep 08 15:41:43.308965 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.308904 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-socket\") pod \"86365d36-8a71-4893-b920-30502e119451\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " Sep 08 15:41:43.309225 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309049 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86365d36-8a71-4893-b920-30502e119451-istio-podinfo\") pod \"86365d36-8a71-4893-b920-30502e119451\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " Sep 08 15:41:43.309225 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309060 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-data" (OuterVolumeSpecName: "istio-data") pod "86365d36-8a71-4893-b920-30502e119451" (UID: "86365d36-8a71-4893-b920-30502e119451"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:41:43.309225 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309178 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "86365d36-8a71-4893-b920-30502e119451" (UID: "86365d36-8a71-4893-b920-30502e119451"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:41:43.309225 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309213 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "86365d36-8a71-4893-b920-30502e119451" (UID: "86365d36-8a71-4893-b920-30502e119451"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:41:43.309421 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309247 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-credential-socket\") pod \"86365d36-8a71-4893-b920-30502e119451\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " Sep 08 15:41:43.309421 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309277 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-envoy\") pod \"86365d36-8a71-4893-b920-30502e119451\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " Sep 08 15:41:43.309421 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309291 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86365d36-8a71-4893-b920-30502e119451-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "86365d36-8a71-4893-b920-30502e119451" (UID: "86365d36-8a71-4893-b920-30502e119451"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:41:43.309421 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309306 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-istio-token\") pod \"86365d36-8a71-4893-b920-30502e119451\" (UID: \"86365d36-8a71-4893-b920-30502e119451\") " Sep 08 15:41:43.309644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309502 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "86365d36-8a71-4893-b920-30502e119451" (UID: "86365d36-8a71-4893-b920-30502e119451"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:41:43.309718 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309691 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:43.309718 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309706 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86365d36-8a71-4893-b920-30502e119451-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:43.309718 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309715 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:43.309870 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309724 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:43.309870 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.309731 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:43.311326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.311302 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86365d36-8a71-4893-b920-30502e119451-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "86365d36-8a71-4893-b920-30502e119451" (UID: "86365d36-8a71-4893-b920-30502e119451"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:41:43.311436 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.311353 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-kube-api-access-vfbwp" (OuterVolumeSpecName: "kube-api-access-vfbwp") pod "86365d36-8a71-4893-b920-30502e119451" (UID: "86365d36-8a71-4893-b920-30502e119451"). InnerVolumeSpecName "kube-api-access-vfbwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:41:43.311483 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.311468 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "86365d36-8a71-4893-b920-30502e119451" (UID: "86365d36-8a71-4893-b920-30502e119451"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:41:43.311755 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.311728 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-istio-token" (OuterVolumeSpecName: "istio-token") pod "86365d36-8a71-4893-b920-30502e119451" (UID: "86365d36-8a71-4893-b920-30502e119451"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:41:43.410903 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.410862 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86365d36-8a71-4893-b920-30502e119451-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:43.411097 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.410897 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86365d36-8a71-4893-b920-30502e119451-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:43.411097 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.410953 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:43.411097 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.410967 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfbwp\" (UniqueName: \"kubernetes.io/projected/86365d36-8a71-4893-b920-30502e119451-kube-api-access-vfbwp\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:41:43.619004 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.618964 2578 generic.go:356] "Generic (PLEG): container finished" podID="86365d36-8a71-4893-b920-30502e119451" containerID="f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe" exitCode=0 Sep 08 15:41:43.619204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.619038 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" Sep 08 15:41:43.619204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.619019 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" event={"ID":"86365d36-8a71-4893-b920-30502e119451","Type":"ContainerDied","Data":"f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe"} Sep 08 15:41:43.619204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.619174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt" event={"ID":"86365d36-8a71-4893-b920-30502e119451","Type":"ContainerDied","Data":"389d1bde2a34bfea80499b13799e56eb6d747db4e600e21daa2c9c31e96dc0ff"} Sep 08 15:41:43.619334 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.619230 2578 scope.go:117] "RemoveContainer" containerID="f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe" Sep 08 15:41:43.631565 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.631545 2578 scope.go:117] "RemoveContainer" containerID="dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045" Sep 08 15:41:43.638726 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.638663 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt"] Sep 08 15:41:43.643611 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.643580 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-630e4d-openshift-default-7565fc8df6-hgmtt"] Sep 08 15:41:43.646300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.646281 2578 scope.go:117] "RemoveContainer" containerID="f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe" Sep 08 15:41:43.646710 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:43.646688 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe\": container with ID starting with f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe not found: ID does not exist" containerID="f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe" Sep 08 15:41:43.646807 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.646722 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe"} err="failed to get container status \"f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe\": rpc error: code = NotFound desc = could not find container \"f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe\": container with ID starting with f7446369016f80ba27875f436dc57aa72ea2814d67d176c4b1cf0168e8fd5bfe not found: ID does not exist" Sep 08 15:41:43.646807 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.646748 2578 scope.go:117] "RemoveContainer" containerID="dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045" Sep 08 15:41:43.647102 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:41:43.647078 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045\": container with ID starting with dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045 not found: ID does not exist" containerID="dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045" Sep 08 15:41:43.647200 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:43.647124 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045"} err="failed to get container status \"dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045\": rpc error: code = NotFound desc = could not find container \"dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045\": container with ID starting with dfca185532fc0b46fa54751089ba219de061eaf886ee9d731dfbbe66a9e65045 not found: ID does not exist" Sep 08 15:41:45.502772 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:45.502734 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86365d36-8a71-4893-b920-30502e119451" path="/var/lib/kubelet/pods/86365d36-8a71-4893-b920-30502e119451/volumes" Sep 08 15:41:54.105897 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:54.105863 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv"] Sep 08 15:41:54.112432 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:54.112397 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bea11f-vjbcv"] Sep 08 15:41:55.502385 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:41:55.502350 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86d4be4d-12de-4010-8478-217518b0a17f" path="/var/lib/kubelet/pods/86d4be4d-12de-4010-8478-217518b0a17f/volumes" Sep 08 15:42:00.818352 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:00.818314 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c"] Sep 08 15:42:00.818834 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:00.818577 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" containerID="cri-o://3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809" gracePeriod=30 Sep 08 15:42:01.225349 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.225313 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2"] Sep 08 15:42:01.225756 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.225744 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d4be4d-12de-4010-8478-217518b0a17f" containerName="revoke-keys" Sep 08 15:42:01.225798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.225760 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d4be4d-12de-4010-8478-217518b0a17f" containerName="revoke-keys" Sep 08 15:42:01.225798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.225783 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" Sep 08 15:42:01.225798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.225788 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" Sep 08 15:42:01.225897 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.225799 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" Sep 08 15:42:01.225897 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.225804 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" Sep 08 15:42:01.225897 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.225896 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" Sep 08 15:42:01.225985 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.225909 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d4be4d-12de-4010-8478-217518b0a17f" containerName="revoke-keys" Sep 08 15:42:01.230582 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.230558 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" Sep 08 15:42:01.234825 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.234797 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:42:01.236143 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.235889 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:42:01.239993 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.239969 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2"] Sep 08 15:42:01.405426 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.405386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8ef8d5-1c58-4fbd-bd68-7f384804f79f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2\" (UID: \"ae8ef8d5-1c58-4fbd-bd68-7f384804f79f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" Sep 08 15:42:01.506536 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.506438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8ef8d5-1c58-4fbd-bd68-7f384804f79f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2\" (UID: \"ae8ef8d5-1c58-4fbd-bd68-7f384804f79f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" Sep 08 15:42:01.507073 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.507048 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8ef8d5-1c58-4fbd-bd68-7f384804f79f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2\" (UID: \"ae8ef8d5-1c58-4fbd-bd68-7f384804f79f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" Sep 08 15:42:01.543414 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.543373 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" Sep 08 15:42:01.830592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:01.830558 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2"] Sep 08 15:42:02.745444 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:02.745406 2578 generic.go:356] "Generic (PLEG): container finished" podID="ae8ef8d5-1c58-4fbd-bd68-7f384804f79f" containerID="2ae5b7f8ab771ec7aceebeb0dbf9f9bc3025f1f561b8303d6bd1446dce997784" exitCode=0 Sep 08 15:42:02.745614 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:02.745482 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" event={"ID":"ae8ef8d5-1c58-4fbd-bd68-7f384804f79f","Type":"ContainerDied","Data":"2ae5b7f8ab771ec7aceebeb0dbf9f9bc3025f1f561b8303d6bd1446dce997784"} Sep 08 15:42:02.745614 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:02.745532 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" event={"ID":"ae8ef8d5-1c58-4fbd-bd68-7f384804f79f","Type":"ContainerStarted","Data":"ea306c5b807217295490b95b6afb36dcb519fd5611b64436a859044f85609158"} Sep 08 15:42:03.893753 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:03.893724 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" Sep 08 15:42:03.928939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:03.928904 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8ef8d5-1c58-4fbd-bd68-7f384804f79f-maas-api-service-ca\") pod \"ae8ef8d5-1c58-4fbd-bd68-7f384804f79f\" (UID: \"ae8ef8d5-1c58-4fbd-bd68-7f384804f79f\") " Sep 08 15:42:03.929346 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:03.929325 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae8ef8d5-1c58-4fbd-bd68-7f384804f79f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae8ef8d5-1c58-4fbd-bd68-7f384804f79f" (UID: "ae8ef8d5-1c58-4fbd-bd68-7f384804f79f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:42:04.030613 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:04.030513 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8ef8d5-1c58-4fbd-bd68-7f384804f79f-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:04.759373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:04.759340 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" event={"ID":"ae8ef8d5-1c58-4fbd-bd68-7f384804f79f","Type":"ContainerDied","Data":"ea306c5b807217295490b95b6afb36dcb519fd5611b64436a859044f85609158"} Sep 08 15:42:04.759550 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:04.759392 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea306c5b807217295490b95b6afb36dcb519fd5611b64436a859044f85609158" Sep 08 15:42:04.759550 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:04.759347 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2" Sep 08 15:42:06.088080 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.088008 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:42:06.152178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152131 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-data\") pod \"193f0014-aaec-45f4-ac12-24c3f2f5f733\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " Sep 08 15:42:06.152178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152172 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-envoy\") pod \"193f0014-aaec-45f4-ac12-24c3f2f5f733\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " Sep 08 15:42:06.152420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152192 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-credential-socket\") pod \"193f0014-aaec-45f4-ac12-24c3f2f5f733\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " Sep 08 15:42:06.152420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152222 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/193f0014-aaec-45f4-ac12-24c3f2f5f733-istiod-ca-cert\") pod \"193f0014-aaec-45f4-ac12-24c3f2f5f733\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " Sep 08 15:42:06.152420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152258 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-token\") pod \"193f0014-aaec-45f4-ac12-24c3f2f5f733\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " Sep 08 15:42:06.152420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152318 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-podinfo\") pod \"193f0014-aaec-45f4-ac12-24c3f2f5f733\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " Sep 08 15:42:06.152420 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152400 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-certs\") pod \"193f0014-aaec-45f4-ac12-24c3f2f5f733\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " Sep 08 15:42:06.152683 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152424 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-data" (OuterVolumeSpecName: "istio-data") pod "193f0014-aaec-45f4-ac12-24c3f2f5f733" (UID: "193f0014-aaec-45f4-ac12-24c3f2f5f733"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:06.152683 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152472 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "193f0014-aaec-45f4-ac12-24c3f2f5f733" (UID: "193f0014-aaec-45f4-ac12-24c3f2f5f733"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:06.152683 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152538 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjkpn\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-kube-api-access-cjkpn\") pod \"193f0014-aaec-45f4-ac12-24c3f2f5f733\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " Sep 08 15:42:06.152683 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152574 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-socket\") pod \"193f0014-aaec-45f4-ac12-24c3f2f5f733\" (UID: \"193f0014-aaec-45f4-ac12-24c3f2f5f733\") " Sep 08 15:42:06.152683 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152629 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "193f0014-aaec-45f4-ac12-24c3f2f5f733" (UID: "193f0014-aaec-45f4-ac12-24c3f2f5f733"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:06.152922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152696 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/193f0014-aaec-45f4-ac12-24c3f2f5f733-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "193f0014-aaec-45f4-ac12-24c3f2f5f733" (UID: "193f0014-aaec-45f4-ac12-24c3f2f5f733"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:42:06.152922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152880 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "193f0014-aaec-45f4-ac12-24c3f2f5f733" (UID: "193f0014-aaec-45f4-ac12-24c3f2f5f733"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:06.153026 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.152998 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:06.153026 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.153019 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:06.153154 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.153032 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:06.153154 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.153045 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:06.153154 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.153062 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/193f0014-aaec-45f4-ac12-24c3f2f5f733-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:06.154590 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.154564 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-token" (OuterVolumeSpecName: "istio-token") pod "193f0014-aaec-45f4-ac12-24c3f2f5f733" (UID: "193f0014-aaec-45f4-ac12-24c3f2f5f733"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:42:06.154734 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.154710 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-kube-api-access-cjkpn" (OuterVolumeSpecName: "kube-api-access-cjkpn") pod "193f0014-aaec-45f4-ac12-24c3f2f5f733" (UID: "193f0014-aaec-45f4-ac12-24c3f2f5f733"). InnerVolumeSpecName "kube-api-access-cjkpn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:42:06.155058 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.155033 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "193f0014-aaec-45f4-ac12-24c3f2f5f733" (UID: "193f0014-aaec-45f4-ac12-24c3f2f5f733"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:06.155058 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.155049 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "193f0014-aaec-45f4-ac12-24c3f2f5f733" (UID: "193f0014-aaec-45f4-ac12-24c3f2f5f733"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:42:06.253874 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.253820 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjkpn\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-kube-api-access-cjkpn\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:06.253874 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.253862 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:06.253874 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.253872 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:06.253874 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.253880 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/193f0014-aaec-45f4-ac12-24c3f2f5f733-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:06.775002 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.774962 2578 generic.go:356] "Generic (PLEG): container finished" podID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerID="3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809" exitCode=0 Sep 08 15:42:06.775222 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.775019 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" Sep 08 15:42:06.775222 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.775050 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" event={"ID":"193f0014-aaec-45f4-ac12-24c3f2f5f733","Type":"ContainerDied","Data":"3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809"} Sep 08 15:42:06.775222 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.775104 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c" event={"ID":"193f0014-aaec-45f4-ac12-24c3f2f5f733","Type":"ContainerDied","Data":"5b308118273a3d399b552d958dbddad752804f36763e3855108459cb4d7c62c0"} Sep 08 15:42:06.775222 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.775149 2578 scope.go:117] "RemoveContainer" containerID="3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809" Sep 08 15:42:06.786069 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.786050 2578 scope.go:117] "RemoveContainer" containerID="73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d" Sep 08 15:42:06.796939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.796911 2578 scope.go:117] "RemoveContainer" containerID="3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809" Sep 08 15:42:06.797322 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:42:06.797295 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809\": container with ID starting with 3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809 not found: ID does not exist" containerID="3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809" Sep 08 15:42:06.797397 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.797333 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809"} err="failed to get container status \"3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809\": rpc error: code = NotFound desc = could not find container \"3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809\": container with ID starting with 3510515c4445ee912918f75cf95577df7cf22ea2dfe33762a2bd2fc713378809 not found: ID does not exist" Sep 08 15:42:06.797397 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.797352 2578 scope.go:117] "RemoveContainer" containerID="73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d" Sep 08 15:42:06.797660 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:42:06.797642 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d\": container with ID starting with 73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d not found: ID does not exist" containerID="73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d" Sep 08 15:42:06.797698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.797667 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d"} err="failed to get container status \"73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d\": rpc error: code = NotFound desc = could not find container \"73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d\": container with ID starting with 73be97745c271a559a5e110faa4df5904d6de71dbc4a0dd26cee8bf0edb9b96d not found: ID does not exist" Sep 08 15:42:06.814964 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.814926 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c"] Sep 08 15:42:06.817851 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:06.817827 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-bea11f-openshift-default-7d9bbd8f46-cgr4c"] Sep 08 15:42:07.506325 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:07.503835 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" path="/var/lib/kubelet/pods/193f0014-aaec-45f4-ac12-24c3f2f5f733/volumes" Sep 08 15:42:12.074698 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:12.074657 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2"] Sep 08 15:42:12.078007 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:12.077974 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-03e8df-bwlr2"] Sep 08 15:42:13.502833 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:13.502798 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae8ef8d5-1c58-4fbd-bd68-7f384804f79f" path="/var/lib/kubelet/pods/ae8ef8d5-1c58-4fbd-bd68-7f384804f79f/volumes" Sep 08 15:42:18.968394 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:18.968354 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd"] Sep 08 15:42:18.968875 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:18.968622 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" containerID="cri-o://82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797" gracePeriod=30 Sep 08 15:42:19.321811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.321708 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2"] Sep 08 15:42:19.322271 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.322254 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" Sep 08 15:42:19.322328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.322276 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" Sep 08 15:42:19.322328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.322296 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" Sep 08 15:42:19.322328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.322302 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" Sep 08 15:42:19.322328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.322308 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae8ef8d5-1c58-4fbd-bd68-7f384804f79f" containerName="revoke-keys" Sep 08 15:42:19.322328 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.322312 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae8ef8d5-1c58-4fbd-bd68-7f384804f79f" containerName="revoke-keys" Sep 08 15:42:19.322495 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.322381 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" Sep 08 15:42:19.322495 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.322392 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="86365d36-8a71-4893-b920-30502e119451" containerName="istio-proxy" Sep 08 15:42:19.322495 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.322400 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae8ef8d5-1c58-4fbd-bd68-7f384804f79f" containerName="revoke-keys" Sep 08 15:42:19.325710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.325693 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" Sep 08 15:42:19.328673 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.328649 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:42:19.328783 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.328699 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:42:19.335092 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.335062 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2"] Sep 08 15:42:19.386596 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.386554 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2\" (UID: \"9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" Sep 08 15:42:19.487841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.487798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2\" (UID: \"9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" Sep 08 15:42:19.488523 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.488501 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2\" (UID: \"9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" Sep 08 15:42:19.638887 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.638796 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" Sep 08 15:42:19.919905 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:19.919841 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2"] Sep 08 15:42:19.922695 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:42:19.922662 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e29bca9_d77d_46c0_8f8c_e5f49a8ab4fe.slice/crio-fce047c450d4eba00bc27cbe045a7b08ff964a68c9169cb4d09efefb39527b60 WatchSource:0}: Error finding container fce047c450d4eba00bc27cbe045a7b08ff964a68c9169cb4d09efefb39527b60: Status 404 returned error can't find the container with id fce047c450d4eba00bc27cbe045a7b08ff964a68c9169cb4d09efefb39527b60 Sep 08 15:42:20.870823 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:20.870784 2578 generic.go:356] "Generic (PLEG): container finished" podID="9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe" containerID="e12cce63c36c2b90ce1e5ae61d5f10d2644a1f59d7e046a82df2a57ecdf8aa09" exitCode=0 Sep 08 15:42:20.871238 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:20.870865 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" event={"ID":"9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe","Type":"ContainerDied","Data":"e12cce63c36c2b90ce1e5ae61d5f10d2644a1f59d7e046a82df2a57ecdf8aa09"} Sep 08 15:42:20.871238 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:20.870938 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" event={"ID":"9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe","Type":"ContainerStarted","Data":"fce047c450d4eba00bc27cbe045a7b08ff964a68c9169cb4d09efefb39527b60"} Sep 08 15:42:22.022219 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:22.022189 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" Sep 08 15:42:22.114759 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:22.114718 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe-maas-api-service-ca\") pod \"9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe\" (UID: \"9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe\") " Sep 08 15:42:22.115099 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:22.115075 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe" (UID: "9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:42:22.215846 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:22.215804 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:22.884729 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:22.884701 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" Sep 08 15:42:22.884904 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:22.884735 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2" event={"ID":"9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe","Type":"ContainerDied","Data":"fce047c450d4eba00bc27cbe045a7b08ff964a68c9169cb4d09efefb39527b60"} Sep 08 15:42:22.884904 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:22.884798 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fce047c450d4eba00bc27cbe045a7b08ff964a68c9169cb4d09efefb39527b60" Sep 08 15:42:23.553338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:23.553295 2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 15:42:23.553719 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:23.553362 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 15:42:24.229185 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.229106 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:42:24.336442 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336400 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-socket\") pod \"770b478b-c229-436e-95b2-166c43fd3724\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " Sep 08 15:42:24.336649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336505 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-istio-token\") pod \"770b478b-c229-436e-95b2-166c43fd3724\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " Sep 08 15:42:24.336649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336526 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-certs\") pod \"770b478b-c229-436e-95b2-166c43fd3724\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " Sep 08 15:42:24.336649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336548 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/770b478b-c229-436e-95b2-166c43fd3724-istio-podinfo\") pod \"770b478b-c229-436e-95b2-166c43fd3724\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " Sep 08 15:42:24.336649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336567 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-data\") pod \"770b478b-c229-436e-95b2-166c43fd3724\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " Sep 08 15:42:24.336649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336598 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/770b478b-c229-436e-95b2-166c43fd3724-istiod-ca-cert\") pod \"770b478b-c229-436e-95b2-166c43fd3724\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " Sep 08 15:42:24.336649 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336625 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfb8l\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-kube-api-access-nfb8l\") pod \"770b478b-c229-436e-95b2-166c43fd3724\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " Sep 08 15:42:24.336950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336667 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-credential-socket\") pod \"770b478b-c229-436e-95b2-166c43fd3724\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " Sep 08 15:42:24.336950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336716 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-envoy\") pod \"770b478b-c229-436e-95b2-166c43fd3724\" (UID: \"770b478b-c229-436e-95b2-166c43fd3724\") " Sep 08 15:42:24.337135 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.336707 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "770b478b-c229-436e-95b2-166c43fd3724" (UID: "770b478b-c229-436e-95b2-166c43fd3724"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:24.337135 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.337046 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "770b478b-c229-436e-95b2-166c43fd3724" (UID: "770b478b-c229-436e-95b2-166c43fd3724"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:24.337326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.337140 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "770b478b-c229-436e-95b2-166c43fd3724" (UID: "770b478b-c229-436e-95b2-166c43fd3724"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:24.337326 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.337171 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-data" (OuterVolumeSpecName: "istio-data") pod "770b478b-c229-436e-95b2-166c43fd3724" (UID: "770b478b-c229-436e-95b2-166c43fd3724"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:24.337586 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.337543 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/770b478b-c229-436e-95b2-166c43fd3724-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "770b478b-c229-436e-95b2-166c43fd3724" (UID: "770b478b-c229-436e-95b2-166c43fd3724"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:42:24.339473 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.339440 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-istio-token" (OuterVolumeSpecName: "istio-token") pod "770b478b-c229-436e-95b2-166c43fd3724" (UID: "770b478b-c229-436e-95b2-166c43fd3724"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:42:24.339473 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.339461 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-kube-api-access-nfb8l" (OuterVolumeSpecName: "kube-api-access-nfb8l") pod "770b478b-c229-436e-95b2-166c43fd3724" (UID: "770b478b-c229-436e-95b2-166c43fd3724"). InnerVolumeSpecName "kube-api-access-nfb8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:42:24.339621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.339530 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/770b478b-c229-436e-95b2-166c43fd3724-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "770b478b-c229-436e-95b2-166c43fd3724" (UID: "770b478b-c229-436e-95b2-166c43fd3724"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:42:24.339621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.339541 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "770b478b-c229-436e-95b2-166c43fd3724" (UID: "770b478b-c229-436e-95b2-166c43fd3724"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:24.438584 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.438540 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:24.438584 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.438579 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:24.438584 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.438591 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:24.438841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.438602 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:24.438841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.438615 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:24.438841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.438625 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/770b478b-c229-436e-95b2-166c43fd3724-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:24.438841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.438634 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/770b478b-c229-436e-95b2-166c43fd3724-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:24.438841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.438644 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/770b478b-c229-436e-95b2-166c43fd3724-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:24.438841 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.438654 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfb8l\" (UniqueName: \"kubernetes.io/projected/770b478b-c229-436e-95b2-166c43fd3724-kube-api-access-nfb8l\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:24.897811 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.897779 2578 generic.go:356] "Generic (PLEG): container finished" podID="770b478b-c229-436e-95b2-166c43fd3724" containerID="82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797" exitCode=0 Sep 08 15:42:24.898250 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.897827 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" Sep 08 15:42:24.898250 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.897854 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" event={"ID":"770b478b-c229-436e-95b2-166c43fd3724","Type":"ContainerDied","Data":"82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797"} Sep 08 15:42:24.898250 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.897896 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd" event={"ID":"770b478b-c229-436e-95b2-166c43fd3724","Type":"ContainerDied","Data":"2eb4e40b67a581ed0c252089fa19b13b2bd609dc7fc2d3bed30b17a44cb377f5"} Sep 08 15:42:24.898250 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.897917 2578 scope.go:117] "RemoveContainer" containerID="82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797" Sep 08 15:42:24.908194 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.908174 2578 scope.go:117] "RemoveContainer" containerID="a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8" Sep 08 15:42:24.917226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.917197 2578 scope.go:117] "RemoveContainer" containerID="82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797" Sep 08 15:42:24.917512 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:42:24.917493 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797\": container with ID starting with 82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797 not found: ID does not exist" containerID="82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797" Sep 08 15:42:24.917568 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.917526 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797"} err="failed to get container status \"82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797\": rpc error: code = NotFound desc = could not find container \"82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797\": container with ID starting with 82d19b6c8038d493f8cdd45709fbdf7add41d3f4f968491add94a056520c3797 not found: ID does not exist" Sep 08 15:42:24.917568 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.917544 2578 scope.go:117] "RemoveContainer" containerID="a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8" Sep 08 15:42:24.917789 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:42:24.917765 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8\": container with ID starting with a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8 not found: ID does not exist" containerID="a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8" Sep 08 15:42:24.917840 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.917801 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8"} err="failed to get container status \"a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8\": rpc error: code = NotFound desc = could not find container \"a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8\": container with ID starting with a1003b1bf1b033eaf8ac444e3b88b2b12a16a4b9066dfe16189e44457d7e82f8 not found: ID does not exist" Sep 08 15:42:24.922700 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.922674 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd"] Sep 08 15:42:24.927395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:24.927372 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-03e8df-openshift-default-cff645bf4-9spkd"] Sep 08 15:42:25.502742 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:25.502705 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="770b478b-c229-436e-95b2-166c43fd3724" path="/var/lib/kubelet/pods/770b478b-c229-436e-95b2-166c43fd3724/volumes" Sep 08 15:42:30.550788 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:30.550745 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2"] Sep 08 15:42:30.555005 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:30.554974 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a7ae4e-9wms2"] Sep 08 15:42:31.502937 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:31.502894 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe" path="/var/lib/kubelet/pods/9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe/volumes" Sep 08 15:42:37.402455 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:37.402413 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6"] Sep 08 15:42:37.402936 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:37.402730 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" containerID="cri-o://9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88" gracePeriod=30 Sep 08 15:42:42.658789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.658764 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:42:42.731221 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731190 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-certs\") pod \"75bb71b2-38c0-4247-b4c4-357feec5c833\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " Sep 08 15:42:42.731221 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731226 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-socket\") pod \"75bb71b2-38c0-4247-b4c4-357feec5c833\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " Sep 08 15:42:42.731458 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731259 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75bb71b2-38c0-4247-b4c4-357feec5c833-istiod-ca-cert\") pod \"75bb71b2-38c0-4247-b4c4-357feec5c833\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " Sep 08 15:42:42.731458 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731319 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-envoy\") pod \"75bb71b2-38c0-4247-b4c4-357feec5c833\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " Sep 08 15:42:42.731458 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731362 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-token\") pod \"75bb71b2-38c0-4247-b4c4-357feec5c833\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " Sep 08 15:42:42.731458 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731384 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfwzq\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-kube-api-access-lfwzq\") pod \"75bb71b2-38c0-4247-b4c4-357feec5c833\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " Sep 08 15:42:42.731458 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731443 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-credential-socket\") pod \"75bb71b2-38c0-4247-b4c4-357feec5c833\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " Sep 08 15:42:42.731708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731539 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75bb71b2-38c0-4247-b4c4-357feec5c833" (UID: "75bb71b2-38c0-4247-b4c4-357feec5c833"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:42.731708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731561 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75bb71b2-38c0-4247-b4c4-357feec5c833" (UID: "75bb71b2-38c0-4247-b4c4-357feec5c833"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:42.731708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731597 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-data\") pod \"75bb71b2-38c0-4247-b4c4-357feec5c833\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " Sep 08 15:42:42.731708 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731632 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-podinfo\") pod \"75bb71b2-38c0-4247-b4c4-357feec5c833\" (UID: \"75bb71b2-38c0-4247-b4c4-357feec5c833\") " Sep 08 15:42:42.731920 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731726 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75bb71b2-38c0-4247-b4c4-357feec5c833" (UID: "75bb71b2-38c0-4247-b4c4-357feec5c833"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:42.731920 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731834 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-data" (OuterVolumeSpecName: "istio-data") pod "75bb71b2-38c0-4247-b4c4-357feec5c833" (UID: "75bb71b2-38c0-4247-b4c4-357feec5c833"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:42.731920 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.731836 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75bb71b2-38c0-4247-b4c4-357feec5c833-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75bb71b2-38c0-4247-b4c4-357feec5c833" (UID: "75bb71b2-38c0-4247-b4c4-357feec5c833"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:42:42.732151 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.732131 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:42.732223 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.732156 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:42.732223 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.732168 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75bb71b2-38c0-4247-b4c4-357feec5c833-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:42.732223 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.732183 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:42.732223 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.732196 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:42.733754 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.733734 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-token" (OuterVolumeSpecName: "istio-token") pod "75bb71b2-38c0-4247-b4c4-357feec5c833" (UID: "75bb71b2-38c0-4247-b4c4-357feec5c833"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:42:42.734188 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.734172 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75bb71b2-38c0-4247-b4c4-357feec5c833" (UID: "75bb71b2-38c0-4247-b4c4-357feec5c833"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:42:42.734278 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.734264 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-kube-api-access-lfwzq" (OuterVolumeSpecName: "kube-api-access-lfwzq") pod "75bb71b2-38c0-4247-b4c4-357feec5c833" (UID: "75bb71b2-38c0-4247-b4c4-357feec5c833"). InnerVolumeSpecName "kube-api-access-lfwzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:42:42.734338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.734323 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75bb71b2-38c0-4247-b4c4-357feec5c833" (UID: "75bb71b2-38c0-4247-b4c4-357feec5c833"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:42:42.833781 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.833743 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:42.833781 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.833784 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:42.833976 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.833801 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfwzq\" (UniqueName: \"kubernetes.io/projected/75bb71b2-38c0-4247-b4c4-357feec5c833-kube-api-access-lfwzq\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:42.833976 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:42.833818 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75bb71b2-38c0-4247-b4c4-357feec5c833-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:42:43.018872 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.018782 2578 generic.go:356] "Generic (PLEG): container finished" podID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerID="9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88" exitCode=0 Sep 08 15:42:43.018872 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.018835 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" Sep 08 15:42:43.019140 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.018870 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" event={"ID":"75bb71b2-38c0-4247-b4c4-357feec5c833","Type":"ContainerDied","Data":"9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88"} Sep 08 15:42:43.019140 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.018935 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6" event={"ID":"75bb71b2-38c0-4247-b4c4-357feec5c833","Type":"ContainerDied","Data":"2c4ca05c3f7e3e3a1897cee3994444bebd190fff3fdb76e76d58b9d6e2924e7e"} Sep 08 15:42:43.019140 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.018964 2578 scope.go:117] "RemoveContainer" containerID="9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88" Sep 08 15:42:43.029892 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.029873 2578 scope.go:117] "RemoveContainer" containerID="84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2" Sep 08 15:42:43.043587 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.043555 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6"] Sep 08 15:42:43.047499 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.047474 2578 scope.go:117] "RemoveContainer" containerID="9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88" Sep 08 15:42:43.047875 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:42:43.047855 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88\": container with ID starting with 9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88 not found: ID does not exist" containerID="9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88" Sep 08 15:42:43.047931 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.047883 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88"} err="failed to get container status \"9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88\": rpc error: code = NotFound desc = could not find container \"9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88\": container with ID starting with 9941012c470a4fe1af946dc021434867a66cd6b4694e73202d234492710e1c88 not found: ID does not exist" Sep 08 15:42:43.047931 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.047902 2578 scope.go:117] "RemoveContainer" containerID="84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2" Sep 08 15:42:43.048155 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.048134 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-a7ae4e-openshift-default-854df7cdbb-55bg6"] Sep 08 15:42:43.048220 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:42:43.048202 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2\": container with ID starting with 84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2 not found: ID does not exist" containerID="84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2" Sep 08 15:42:43.048263 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.048226 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2"} err="failed to get container status \"84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2\": rpc error: code = NotFound desc = could not find container \"84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2\": container with ID starting with 84ff2826491a99b217e8cb017158567a76d64a326e9885c949bef7216f20f2f2 not found: ID does not exist" Sep 08 15:42:43.502950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:42:43.502913 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" path="/var/lib/kubelet/pods/75bb71b2-38c0-4247-b4c4-357feec5c833/volumes" Sep 08 15:43:10.075652 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.075572 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw"] Sep 08 15:43:10.076071 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076053 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076074 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076097 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076103 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076128 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076134 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076140 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076146 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076152 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe" containerName="revoke-keys" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076157 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe" containerName="revoke-keys" Sep 08 15:43:10.076226 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076222 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" Sep 08 15:43:10.076535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076232 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" Sep 08 15:43:10.076535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076238 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="193f0014-aaec-45f4-ac12-24c3f2f5f733" containerName="istio-proxy" Sep 08 15:43:10.076535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076245 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e29bca9-d77d-46c0-8f8c-e5f49a8ab4fe" containerName="revoke-keys" Sep 08 15:43:10.076535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076252 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="770b478b-c229-436e-95b2-166c43fd3724" containerName="istio-proxy" Sep 08 15:43:10.076535 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.076258 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="75bb71b2-38c0-4247-b4c4-357feec5c833" containerName="istio-proxy" Sep 08 15:43:10.081092 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.081075 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.086466 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.086446 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-41de51-openshift-default-dockercfg-8wcmt\"" Sep 08 15:43:10.098707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.098670 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw"] Sep 08 15:43:10.209520 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.209485 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-token\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.209716 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.209555 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2694v\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-kube-api-access-2694v\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.209716 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.209577 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-data\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.209716 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.209600 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istiod-ca-cert\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.209716 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.209639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-socket\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.209716 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.209703 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-credential-socket\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.209940 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.209750 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-envoy\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.209940 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.209796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-certs\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.209940 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.209852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-podinfo\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311195 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311151 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-socket\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311195 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-credential-socket\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-envoy\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311244 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-certs\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-podinfo\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-token\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2694v\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-kube-api-access-2694v\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-data\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311457 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istiod-ca-cert\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311637 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-socket\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311697 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-credential-socket\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311758 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-certs\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.311973 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.311851 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-data\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.312504 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.312481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istiod-ca-cert\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.313803 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.313773 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-envoy\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.313911 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.313893 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-podinfo\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.323399 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.323314 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2694v\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-kube-api-access-2694v\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.325023 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.324999 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-token\") pod \"e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.391593 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.391498 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:10.676450 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.676418 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw"] Sep 08 15:43:10.677381 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:43:10.677352 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9c44f8_661d_418c_88ad_0cf49eb075f8.slice/crio-4388ce87421a9f03a329ddba5c0207238ecefce9d56ab0f468cd538928f37a62 WatchSource:0}: Error finding container 4388ce87421a9f03a329ddba5c0207238ecefce9d56ab0f468cd538928f37a62: Status 404 returned error can't find the container with id 4388ce87421a9f03a329ddba5c0207238ecefce9d56ab0f468cd538928f37a62 Sep 08 15:43:10.679639 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.679611 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:10.679710 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.679678 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:10.679752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:10.679707 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:11.196187 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:11.196146 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" event={"ID":"fb9c44f8-661d-418c-88ad-0cf49eb075f8","Type":"ContainerStarted","Data":"dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7"} Sep 08 15:43:11.196588 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:11.196219 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" event={"ID":"fb9c44f8-661d-418c-88ad-0cf49eb075f8","Type":"ContainerStarted","Data":"4388ce87421a9f03a329ddba5c0207238ecefce9d56ab0f468cd538928f37a62"} Sep 08 15:43:11.220552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:11.220494 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" podStartSLOduration=1.220477299 podStartE2EDuration="1.220477299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:43:11.218639182 +0000 UTC m=+1530.242263639" watchObservedRunningTime="2026-09-08 15:43:11.220477299 +0000 UTC m=+1530.244101755" Sep 08 15:43:11.392224 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:11.392172 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:11.397611 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:11.397580 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:12.203750 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:12.203720 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:12.204901 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:12.204885 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:27.926254 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:27.925371 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": read tcp 10.134.0.2:57092->10.134.0.85:15021: read: connection reset by peer" start-of-body= Sep 08 15:43:27.926254 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:27.925455 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": read tcp 10.134.0.2:57092->10.134.0.85:15021: read: connection reset by peer" Sep 08 15:43:28.307744 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:28.307710 2578 generic.go:356] "Generic (PLEG): container finished" podID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerID="dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7" exitCode=0 Sep 08 15:43:28.307926 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:28.307770 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" event={"ID":"fb9c44f8-661d-418c-88ad-0cf49eb075f8","Type":"ContainerDied","Data":"dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7"} Sep 08 15:43:28.308254 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:28.308234 2578 scope.go:117] "RemoveContainer" containerID="dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7" Sep 08 15:43:28.309357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:28.309324 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:28.309467 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:28.309410 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:28.309467 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:28.309460 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:29.315432 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:29.315390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" event={"ID":"fb9c44f8-661d-418c-88ad-0cf49eb075f8","Type":"ContainerStarted","Data":"a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb"} Sep 08 15:43:29.392216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:29.392174 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:29.392397 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:29.392374 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Sep 08 15:43:29.392453 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:29.392423 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Sep 08 15:43:30.392635 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:30.392600 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Sep 08 15:43:30.393063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:30.392662 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Sep 08 15:43:31.392361 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:31.392324 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Sep 08 15:43:31.392559 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:31.392383 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Sep 08 15:43:32.392839 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:32.392798 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Sep 08 15:43:32.393290 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:32.392859 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Sep 08 15:43:33.391891 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:33.391853 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Sep 08 15:43:33.392077 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:33.391909 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Sep 08 15:43:34.391967 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:34.391929 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Sep 08 15:43:34.392405 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:34.391988 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Sep 08 15:43:35.396151 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:35.396119 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:35.396529 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:35.396506 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:35.397268 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:35.397252 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:43:39.013610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.013575 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4"] Sep 08 15:43:39.018061 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.018038 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.021958 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.021933 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-0fea02-openshift-default-dockercfg-8kshw\"" Sep 08 15:43:39.033436 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.033404 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4"] Sep 08 15:43:39.114601 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.114554 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-envoy\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.114814 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.114623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-socket\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.114814 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.114673 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-token\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.114814 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.114751 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-credential-socket\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.115000 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.114814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-certs\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.115000 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.114860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-podinfo\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.115000 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.114890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-data\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.115000 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.114947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc742a6-be92-4dac-a3b4-a311aa96a668-istiod-ca-cert\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.115000 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.114963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv5bl\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-kube-api-access-rv5bl\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.215569 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.215530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-credential-socket\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.215569 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.215577 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-certs\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.215801 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.215606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-podinfo\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.215801 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.215737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-data\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.215801 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.215791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc742a6-be92-4dac-a3b4-a311aa96a668-istiod-ca-cert\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.215966 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.215814 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv5bl\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-kube-api-access-rv5bl\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.216033 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.216006 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-credential-socket\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.216033 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.216015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-envoy\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.216205 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.216078 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-certs\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.216205 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.216160 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-data\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.216205 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.216105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-socket\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.216342 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.216255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-token\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.216507 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.216430 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-socket\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.216581 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.216546 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc742a6-be92-4dac-a3b4-a311aa96a668-istiod-ca-cert\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.218282 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.218259 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-envoy\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.218380 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.218338 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-podinfo\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.226720 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.226684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-token\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.226862 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.226828 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv5bl\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-kube-api-access-rv5bl\") pod \"e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.329257 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.329166 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:39.613237 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.613212 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4"] Sep 08 15:43:39.615052 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:43:39.615022 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc742a6_be92_4dac_a3b4_a311aa96a668.slice/crio-24b88d08f3d1216b5458cf2dc37fda8b62e712538d1472fda211b8c46eea00e1 WatchSource:0}: Error finding container 24b88d08f3d1216b5458cf2dc37fda8b62e712538d1472fda211b8c46eea00e1: Status 404 returned error can't find the container with id 24b88d08f3d1216b5458cf2dc37fda8b62e712538d1472fda211b8c46eea00e1 Sep 08 15:43:39.617336 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.617305 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:39.617438 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.617367 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:39.617438 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:39.617395 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:40.406178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:40.406144 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" event={"ID":"2bc742a6-be92-4dac-a3b4-a311aa96a668","Type":"ContainerStarted","Data":"e12fab4c59617b5594f4471bb7a989ecf33716e03405d95446987ccd35ca33c6"} Sep 08 15:43:40.406725 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:40.406206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" event={"ID":"2bc742a6-be92-4dac-a3b4-a311aa96a668","Type":"ContainerStarted","Data":"24b88d08f3d1216b5458cf2dc37fda8b62e712538d1472fda211b8c46eea00e1"} Sep 08 15:43:40.435052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:40.434995 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" podStartSLOduration=2.4349772610000002 podStartE2EDuration="2.434977261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:43:40.433084024 +0000 UTC m=+1559.456708485" watchObservedRunningTime="2026-09-08 15:43:40.434977261 +0000 UTC m=+1559.458601717" Sep 08 15:43:41.330204 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:41.330164 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:41.335569 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:41.335543 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:41.414178 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:41.414149 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:41.415151 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:41.415100 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:43:57.456305 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:57.456249 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 08 15:43:57.456847 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:57.456317 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 15:43:58.524953 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:58.524875 2578 generic.go:356] "Generic (PLEG): container finished" podID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerID="e12fab4c59617b5594f4471bb7a989ecf33716e03405d95446987ccd35ca33c6" exitCode=0 Sep 08 15:43:58.525360 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:58.524953 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" event={"ID":"2bc742a6-be92-4dac-a3b4-a311aa96a668","Type":"ContainerDied","Data":"e12fab4c59617b5594f4471bb7a989ecf33716e03405d95446987ccd35ca33c6"} Sep 08 15:43:58.525429 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:58.525411 2578 scope.go:117] "RemoveContainer" containerID="e12fab4c59617b5594f4471bb7a989ecf33716e03405d95446987ccd35ca33c6" Sep 08 15:43:58.526657 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:58.526612 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:58.526719 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:58.526707 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:58.526756 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:58.526742 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:43:59.537772 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:43:59.537724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" event={"ID":"2bc742a6-be92-4dac-a3b4-a311aa96a668","Type":"ContainerStarted","Data":"383bcb824586ebe48b988b3639efb7fdecf043c9ea8bca33195bb5355611bc1e"} Sep 08 15:44:00.329766 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:00.329618 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:44:00.329999 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:00.329917 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 08 15:44:00.330073 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:00.329984 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 08 15:44:01.330381 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:01.330330 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 08 15:44:01.330913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:01.330406 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 08 15:44:02.330526 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:02.330477 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 08 15:44:02.331152 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:02.331097 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 08 15:44:03.330131 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:03.330064 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 08 15:44:03.330438 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:03.330404 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 08 15:44:04.329693 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:04.329650 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 08 15:44:04.330325 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:04.330291 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 08 15:44:05.333957 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:05.333921 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:44:05.334383 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:05.334203 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:44:05.334928 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:05.334908 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:44:41.884464 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:41.884385 2578 scope.go:117] "RemoveContainer" containerID="7c56b29a5f87ed3fde3c3a22745df4430f25b05f05f20c08ff0772480b62a18b" Sep 08 15:44:51.321895 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.321854 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2"] Sep 08 15:44:51.326963 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.326944 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.329742 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.329715 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-21c704-openshift-default-dockercfg-25drw\"" Sep 08 15:44:51.340050 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.340021 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2"] Sep 08 15:44:51.472866 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.472794 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-socket\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.473089 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.472931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-credential-socket\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.473089 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.473002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-envoy\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.473089 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.473077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0de597-8fc8-48ce-b779-6f4936729811-istiod-ca-cert\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.473300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.473202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-istio-token\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.473357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.473296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0de597-8fc8-48ce-b779-6f4936729811-istio-podinfo\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.473415 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.473355 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bx7n\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-kube-api-access-4bx7n\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.473415 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.473388 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-data\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.473538 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.473457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-certs\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.574823 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.574733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-credential-socket\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.574823 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.574771 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-envoy\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.574823 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.574807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0de597-8fc8-48ce-b779-6f4936729811-istiod-ca-cert\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.574855 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-istio-token\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.574885 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0de597-8fc8-48ce-b779-6f4936729811-istio-podinfo\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.574919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bx7n\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-kube-api-access-4bx7n\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.574948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-data\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.575011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-certs\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.575070 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-socket\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575489 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.575223 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-credential-socket\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575550 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.575525 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-certs\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.575545 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-data\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.575595 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0de597-8fc8-48ce-b779-6f4936729811-istiod-ca-cert\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.575717 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.575665 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-socket\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.577363 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.577345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-envoy\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.577509 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.577491 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0de597-8fc8-48ce-b779-6f4936729811-istio-podinfo\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.585602 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.585571 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-istio-token\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.585751 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.585640 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bx7n\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-kube-api-access-4bx7n\") pod \"e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.637702 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.637660 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:51.923893 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.923856 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2"] Sep 08 15:44:51.925585 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:44:51.925560 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0de597_8fc8_48ce_b779_6f4936729811.slice/crio-b6a671475645ac195c0e93dd689279ae90c79b06e7b815860cc4b78978981071 WatchSource:0}: Error finding container b6a671475645ac195c0e93dd689279ae90c79b06e7b815860cc4b78978981071: Status 404 returned error can't find the container with id b6a671475645ac195c0e93dd689279ae90c79b06e7b815860cc4b78978981071 Sep 08 15:44:51.928280 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.928248 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:44:51.928376 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.928319 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:44:51.928376 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:51.928349 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:44:52.889198 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:52.889158 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" event={"ID":"0e0de597-8fc8-48ce-b779-6f4936729811","Type":"ContainerStarted","Data":"a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb"} Sep 08 15:44:52.889707 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:52.889216 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" event={"ID":"0e0de597-8fc8-48ce-b779-6f4936729811","Type":"ContainerStarted","Data":"b6a671475645ac195c0e93dd689279ae90c79b06e7b815860cc4b78978981071"} Sep 08 15:44:52.915391 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:52.915335 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" podStartSLOduration=1.9153185590000001 podStartE2EDuration="1.915318559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:44:52.914011913 +0000 UTC m=+1631.937636369" watchObservedRunningTime="2026-09-08 15:44:52.915318559 +0000 UTC m=+1631.938943015" Sep 08 15:44:53.638828 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:53.638788 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:53.644568 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:53.644544 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:53.895714 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:53.895638 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:44:53.896583 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:44:53.896564 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:45:00.152650 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.152609 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt"] Sep 08 15:45:00.157142 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.157097 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525"] Sep 08 15:45:00.157319 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.157208 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt" Sep 08 15:45:00.161039 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.160858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525" Sep 08 15:45:00.162828 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.162797 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:45:00.166167 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.166144 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6"] Sep 08 15:45:00.169924 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.169906 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6" Sep 08 15:45:00.170885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.170861 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8"] Sep 08 15:45:00.173126 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.173082 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt" Sep 08 15:45:00.175367 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.175347 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8" Sep 08 15:45:00.175901 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.175880 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt"] Sep 08 15:45:00.186516 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.186479 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525"] Sep 08 15:45:00.189807 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.189786 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525" Sep 08 15:45:00.191862 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.191829 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6"] Sep 08 15:45:00.196789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.196756 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8"] Sep 08 15:45:00.243631 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.243600 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6" Sep 08 15:45:00.253700 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.253385 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8" Sep 08 15:45:00.474821 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.474789 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt"] Sep 08 15:45:00.476154 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:45:00.476100 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b39e337_be08_4753_9c2a_b80c2eecb367.slice/crio-4d39627fbb8031f926c6a7d7fb8ca25d6c6d5500e3d615467b84e44a0cae8a67 WatchSource:0}: Error finding container 4d39627fbb8031f926c6a7d7fb8ca25d6c6d5500e3d615467b84e44a0cae8a67: Status 404 returned error can't find the container with id 4d39627fbb8031f926c6a7d7fb8ca25d6c6d5500e3d615467b84e44a0cae8a67 Sep 08 15:45:00.502550 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.502522 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525"] Sep 08 15:45:00.512385 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:45:00.512357 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c85a31a_0f29_48f7_ae85_95b24c9543b2.slice/crio-3874c4c5bcbae1094180ca4ed879c9ed16cf971fffbb7f1796d7626fe4403ba1 WatchSource:0}: Error finding container 3874c4c5bcbae1094180ca4ed879c9ed16cf971fffbb7f1796d7626fe4403ba1: Status 404 returned error can't find the container with id 3874c4c5bcbae1094180ca4ed879c9ed16cf971fffbb7f1796d7626fe4403ba1 Sep 08 15:45:00.557922 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.557892 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6"] Sep 08 15:45:00.558404 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:45:00.558379 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17be8490_19c1_4bda_b43e_ed5de68290ad.slice/crio-ee68d988a85cc543e9e1f5790e39c9140e204034d2f73fe65be21ec8f57a2865 WatchSource:0}: Error finding container ee68d988a85cc543e9e1f5790e39c9140e204034d2f73fe65be21ec8f57a2865: Status 404 returned error can't find the container with id ee68d988a85cc543e9e1f5790e39c9140e204034d2f73fe65be21ec8f57a2865 Sep 08 15:45:00.564165 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.564136 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8"] Sep 08 15:45:00.565364 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:45:00.565340 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69792bb7_1f80_4427_8490_0e73fd3a19e4.slice/crio-1e315a22391411ffc1ae5ec70fb08a3bf998f5ea1b20ad129e328668669928a5 WatchSource:0}: Error finding container 1e315a22391411ffc1ae5ec70fb08a3bf998f5ea1b20ad129e328668669928a5: Status 404 returned error can't find the container with id 1e315a22391411ffc1ae5ec70fb08a3bf998f5ea1b20ad129e328668669928a5 Sep 08 15:45:00.942357 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.942328 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6" event={"ID":"17be8490-19c1-4bda-b43e-ed5de68290ad","Type":"ContainerStarted","Data":"ee68d988a85cc543e9e1f5790e39c9140e204034d2f73fe65be21ec8f57a2865"} Sep 08 15:45:00.944071 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.944045 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt" event={"ID":"3b39e337-be08-4753-9c2a-b80c2eecb367","Type":"ContainerStarted","Data":"4e9834436cfcf67dbc85d831495323e5b27f9eeb136e68a6a44b4ec32d903b9c"} Sep 08 15:45:00.944212 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.944103 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt" event={"ID":"3b39e337-be08-4753-9c2a-b80c2eecb367","Type":"ContainerStarted","Data":"4d39627fbb8031f926c6a7d7fb8ca25d6c6d5500e3d615467b84e44a0cae8a67"} Sep 08 15:45:00.945679 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.945632 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525" event={"ID":"7c85a31a-0f29-48f7-ae85-95b24c9543b2","Type":"ContainerStarted","Data":"8a13d607cdb242b7730054529e60f19fb4c99c9b9047b4c8e5d5d25d597a581a"} Sep 08 15:45:00.945783 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.945693 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525" event={"ID":"7c85a31a-0f29-48f7-ae85-95b24c9543b2","Type":"ContainerStarted","Data":"3874c4c5bcbae1094180ca4ed879c9ed16cf971fffbb7f1796d7626fe4403ba1"} Sep 08 15:45:00.947242 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.947218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8" event={"ID":"69792bb7-1f80-4427-8490-0e73fd3a19e4","Type":"ContainerStarted","Data":"1e315a22391411ffc1ae5ec70fb08a3bf998f5ea1b20ad129e328668669928a5"} Sep 08 15:45:00.964550 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.964482 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt" podStartSLOduration=0.964464865 podStartE2EDuration="964.464865ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:00.962298589 +0000 UTC m=+1639.985923057" watchObservedRunningTime="2026-09-08 15:45:00.964464865 +0000 UTC m=+1639.988089321" Sep 08 15:45:00.981222 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.981163 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525" podStartSLOduration=0.981144517 podStartE2EDuration="981.144517ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:00.979269184 +0000 UTC m=+1640.002893639" watchObservedRunningTime="2026-09-08 15:45:00.981144517 +0000 UTC m=+1640.004769255" Sep 08 15:45:00.998150 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:00.998032 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8" podStartSLOduration=0.998014918 podStartE2EDuration="998.014918ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:00.996385248 +0000 UTC m=+1640.020009703" watchObservedRunningTime="2026-09-08 15:45:00.998014918 +0000 UTC m=+1640.021639375" Sep 08 15:45:01.955939 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:01.955904 2578 generic.go:356] "Generic (PLEG): container finished" podID="17be8490-19c1-4bda-b43e-ed5de68290ad" containerID="6cbb83c28d27ee2cf60e6987a8487b528d079f1c94bbaa8dbb7f907c37545465" exitCode=0 Sep 08 15:45:01.956409 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:01.955993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6" event={"ID":"17be8490-19c1-4bda-b43e-ed5de68290ad","Type":"ContainerDied","Data":"6cbb83c28d27ee2cf60e6987a8487b528d079f1c94bbaa8dbb7f907c37545465"} Sep 08 15:45:01.957662 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:01.957639 2578 generic.go:356] "Generic (PLEG): container finished" podID="3b39e337-be08-4753-9c2a-b80c2eecb367" containerID="4e9834436cfcf67dbc85d831495323e5b27f9eeb136e68a6a44b4ec32d903b9c" exitCode=0 Sep 08 15:45:01.957794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:01.957712 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt" event={"ID":"3b39e337-be08-4753-9c2a-b80c2eecb367","Type":"ContainerDied","Data":"4e9834436cfcf67dbc85d831495323e5b27f9eeb136e68a6a44b4ec32d903b9c"} Sep 08 15:45:01.959304 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:01.959284 2578 generic.go:356] "Generic (PLEG): container finished" podID="7c85a31a-0f29-48f7-ae85-95b24c9543b2" containerID="8a13d607cdb242b7730054529e60f19fb4c99c9b9047b4c8e5d5d25d597a581a" exitCode=0 Sep 08 15:45:01.959383 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:01.959365 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525" event={"ID":"7c85a31a-0f29-48f7-ae85-95b24c9543b2","Type":"ContainerDied","Data":"8a13d607cdb242b7730054529e60f19fb4c99c9b9047b4c8e5d5d25d597a581a"} Sep 08 15:45:01.960832 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:01.960811 2578 generic.go:356] "Generic (PLEG): container finished" podID="69792bb7-1f80-4427-8490-0e73fd3a19e4" containerID="bdc3ae814e3fe80b367952bad345426ecd3857642ab64cfa5ff0f2a9169b70ba" exitCode=0 Sep 08 15:45:01.960900 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:01.960870 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8" event={"ID":"69792bb7-1f80-4427-8490-0e73fd3a19e4","Type":"ContainerDied","Data":"bdc3ae814e3fe80b367952bad345426ecd3857642ab64cfa5ff0f2a9169b70ba"} Sep 08 15:45:03.151043 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.151010 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt" Sep 08 15:45:03.154344 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.154313 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8" Sep 08 15:45:03.199489 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.199458 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6" Sep 08 15:45:03.203043 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.203015 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525" Sep 08 15:45:03.977621 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.977586 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6" Sep 08 15:45:03.977842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.977616 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6" event={"ID":"17be8490-19c1-4bda-b43e-ed5de68290ad","Type":"ContainerDied","Data":"ee68d988a85cc543e9e1f5790e39c9140e204034d2f73fe65be21ec8f57a2865"} Sep 08 15:45:03.977842 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.977696 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee68d988a85cc543e9e1f5790e39c9140e204034d2f73fe65be21ec8f57a2865" Sep 08 15:45:03.979269 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.979249 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt" Sep 08 15:45:03.979373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.979262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt" event={"ID":"3b39e337-be08-4753-9c2a-b80c2eecb367","Type":"ContainerDied","Data":"4d39627fbb8031f926c6a7d7fb8ca25d6c6d5500e3d615467b84e44a0cae8a67"} Sep 08 15:45:03.979373 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.979310 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d39627fbb8031f926c6a7d7fb8ca25d6c6d5500e3d615467b84e44a0cae8a67" Sep 08 15:45:03.980819 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.980802 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525" Sep 08 15:45:03.980934 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.980843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525" event={"ID":"7c85a31a-0f29-48f7-ae85-95b24c9543b2","Type":"ContainerDied","Data":"3874c4c5bcbae1094180ca4ed879c9ed16cf971fffbb7f1796d7626fe4403ba1"} Sep 08 15:45:03.980934 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.980888 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3874c4c5bcbae1094180ca4ed879c9ed16cf971fffbb7f1796d7626fe4403ba1" Sep 08 15:45:03.982824 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.982805 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8" Sep 08 15:45:03.982909 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.982803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8" event={"ID":"69792bb7-1f80-4427-8490-0e73fd3a19e4","Type":"ContainerDied","Data":"1e315a22391411ffc1ae5ec70fb08a3bf998f5ea1b20ad129e328668669928a5"} Sep 08 15:45:03.982959 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:03.982920 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e315a22391411ffc1ae5ec70fb08a3bf998f5ea1b20ad129e328668669928a5" Sep 08 15:45:14.049650 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:14.049617 2578 generic.go:356] "Generic (PLEG): container finished" podID="0e0de597-8fc8-48ce-b779-6f4936729811" containerID="a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb" exitCode=0 Sep 08 15:45:14.050087 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:14.049689 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" event={"ID":"0e0de597-8fc8-48ce-b779-6f4936729811","Type":"ContainerDied","Data":"a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb"} Sep 08 15:45:14.050164 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:14.050150 2578 scope.go:117] "RemoveContainer" containerID="a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb" Sep 08 15:45:14.051415 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:14.051388 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:45:14.051490 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:14.051457 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:45:14.051528 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:14.051497 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 15:45:15.058610 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:15.058568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" event={"ID":"0e0de597-8fc8-48ce-b779-6f4936729811","Type":"ContainerStarted","Data":"6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa"} Sep 08 15:45:15.638888 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:15.638851 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:45:15.639149 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:15.639089 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body= Sep 08 15:45:15.639201 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:15.639178 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" Sep 08 15:45:16.638220 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:16.638176 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body= Sep 08 15:45:16.638764 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:16.638260 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" Sep 08 15:45:17.638597 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:17.638555 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body= Sep 08 15:45:17.639209 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:17.639177 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" Sep 08 15:45:18.642143 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:18.638549 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body= Sep 08 15:45:18.642143 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:18.638615 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" Sep 08 15:45:19.643645 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:19.642641 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body= Sep 08 15:45:19.643645 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:19.642726 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" Sep 08 15:45:20.576232 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576194 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s"] Sep 08 15:45:20.576645 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576633 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b39e337-be08-4753-9c2a-b80c2eecb367" containerName="cleanup" Sep 08 15:45:20.576689 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576647 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b39e337-be08-4753-9c2a-b80c2eecb367" containerName="cleanup" Sep 08 15:45:20.576689 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576678 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69792bb7-1f80-4427-8490-0e73fd3a19e4" containerName="cleanup" Sep 08 15:45:20.576689 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576685 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="69792bb7-1f80-4427-8490-0e73fd3a19e4" containerName="cleanup" Sep 08 15:45:20.576783 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576693 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c85a31a-0f29-48f7-ae85-95b24c9543b2" containerName="cleanup" Sep 08 15:45:20.576783 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576698 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c85a31a-0f29-48f7-ae85-95b24c9543b2" containerName="cleanup" Sep 08 15:45:20.576783 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576704 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17be8490-19c1-4bda-b43e-ed5de68290ad" containerName="cleanup" Sep 08 15:45:20.576783 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576709 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="17be8490-19c1-4bda-b43e-ed5de68290ad" containerName="cleanup" Sep 08 15:45:20.576783 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576776 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b39e337-be08-4753-9c2a-b80c2eecb367" containerName="cleanup" Sep 08 15:45:20.576933 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576786 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="69792bb7-1f80-4427-8490-0e73fd3a19e4" containerName="cleanup" Sep 08 15:45:20.576933 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576793 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="17be8490-19c1-4bda-b43e-ed5de68290ad" containerName="cleanup" Sep 08 15:45:20.576933 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.576800 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c85a31a-0f29-48f7-ae85-95b24c9543b2" containerName="cleanup" Sep 08 15:45:20.580143 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.580097 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" Sep 08 15:45:20.583232 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.583202 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:45:20.583424 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.583325 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:45:20.591149 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.591090 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s"] Sep 08 15:45:20.642139 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.641325 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body= Sep 08 15:45:20.642139 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.641411 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" Sep 08 15:45:20.684286 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.684238 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160567e-d143-4c87-906c-9f6a5b61e8fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s\" (UID: \"d160567e-d143-4c87-906c-9f6a5b61e8fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" Sep 08 15:45:20.785959 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.785913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160567e-d143-4c87-906c-9f6a5b61e8fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s\" (UID: \"d160567e-d143-4c87-906c-9f6a5b61e8fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" Sep 08 15:45:20.786615 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.786591 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160567e-d143-4c87-906c-9f6a5b61e8fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s\" (UID: \"d160567e-d143-4c87-906c-9f6a5b61e8fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" Sep 08 15:45:20.893663 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:20.893562 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" Sep 08 15:45:21.199398 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:21.199352 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s"] Sep 08 15:45:21.200227 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:45:21.200196 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd160567e_d143_4c87_906c_9f6a5b61e8fe.slice/crio-12e66a4fd390976705f3a6e10975d033e90a7aae2241b62a2e73a1d8dbc73460 WatchSource:0}: Error finding container 12e66a4fd390976705f3a6e10975d033e90a7aae2241b62a2e73a1d8dbc73460: Status 404 returned error can't find the container with id 12e66a4fd390976705f3a6e10975d033e90a7aae2241b62a2e73a1d8dbc73460 Sep 08 15:45:21.684861 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:21.684827 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:45:21.685358 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:21.685142 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:45:21.685994 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:21.685974 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:45:22.107914 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:22.107879 2578 generic.go:356] "Generic (PLEG): container finished" podID="d160567e-d143-4c87-906c-9f6a5b61e8fe" containerID="e68034e30f4a73d3c161e0e8d004c35bdd39afae98780c930c0a439de406d12b" exitCode=0 Sep 08 15:45:22.108103 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:22.107963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" event={"ID":"d160567e-d143-4c87-906c-9f6a5b61e8fe","Type":"ContainerDied","Data":"e68034e30f4a73d3c161e0e8d004c35bdd39afae98780c930c0a439de406d12b"} Sep 08 15:45:22.108103 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:22.108019 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" event={"ID":"d160567e-d143-4c87-906c-9f6a5b61e8fe","Type":"ContainerStarted","Data":"12e66a4fd390976705f3a6e10975d033e90a7aae2241b62a2e73a1d8dbc73460"} Sep 08 15:45:23.250676 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:23.250652 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" Sep 08 15:45:23.312007 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:23.311975 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160567e-d143-4c87-906c-9f6a5b61e8fe-maas-api-service-ca\") pod \"d160567e-d143-4c87-906c-9f6a5b61e8fe\" (UID: \"d160567e-d143-4c87-906c-9f6a5b61e8fe\") " Sep 08 15:45:23.312333 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:23.312313 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d160567e-d143-4c87-906c-9f6a5b61e8fe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d160567e-d143-4c87-906c-9f6a5b61e8fe" (UID: "d160567e-d143-4c87-906c-9f6a5b61e8fe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:45:23.413874 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:23.413776 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160567e-d143-4c87-906c-9f6a5b61e8fe-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:24.131164 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:24.131127 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" Sep 08 15:45:24.131164 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:24.131144 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s" event={"ID":"d160567e-d143-4c87-906c-9f6a5b61e8fe","Type":"ContainerDied","Data":"12e66a4fd390976705f3a6e10975d033e90a7aae2241b62a2e73a1d8dbc73460"} Sep 08 15:45:24.131389 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:24.131222 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12e66a4fd390976705f3a6e10975d033e90a7aae2241b62a2e73a1d8dbc73460" Sep 08 15:45:26.899102 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:26.899064 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s"] Sep 08 15:45:26.902087 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:26.902058 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-21c704-m6r2s"] Sep 08 15:45:27.502495 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:27.502453 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d160567e-d143-4c87-906c-9f6a5b61e8fe" path="/var/lib/kubelet/pods/d160567e-d143-4c87-906c-9f6a5b61e8fe/volumes" Sep 08 15:45:33.732279 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:33.732243 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2"] Sep 08 15:45:33.732748 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:33.732498 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" containerID="cri-o://6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa" gracePeriod=30 Sep 08 15:45:36.687441 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:36.687397 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 15:45:36.687815 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:36.687479 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 15:45:37.307997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.307960 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q"] Sep 08 15:45:37.308451 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.308437 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d160567e-d143-4c87-906c-9f6a5b61e8fe" containerName="revoke-keys" Sep 08 15:45:37.308501 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.308454 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d160567e-d143-4c87-906c-9f6a5b61e8fe" containerName="revoke-keys" Sep 08 15:45:37.308553 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.308544 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d160567e-d143-4c87-906c-9f6a5b61e8fe" containerName="revoke-keys" Sep 08 15:45:37.312863 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.312843 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" Sep 08 15:45:37.315630 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.315598 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:45:37.315752 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.315639 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:45:37.322338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.322311 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q"] Sep 08 15:45:37.458531 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.458493 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q\" (UID: \"f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" Sep 08 15:45:37.560191 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.560072 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q\" (UID: \"f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" Sep 08 15:45:37.560728 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.560708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q\" (UID: \"f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" Sep 08 15:45:37.624911 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.624866 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" Sep 08 15:45:37.905324 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:37.905292 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q"] Sep 08 15:45:37.906222 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:45:37.906192 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1eea4d9_656b_4aaf_b8f1_e7e3d77510d4.slice/crio-fcc15fa5ced7c5e7021088a2425f5cf914934bb8896727a8ab2dcbe231bcc654 WatchSource:0}: Error finding container fcc15fa5ced7c5e7021088a2425f5cf914934bb8896727a8ab2dcbe231bcc654: Status 404 returned error can't find the container with id fcc15fa5ced7c5e7021088a2425f5cf914934bb8896727a8ab2dcbe231bcc654 Sep 08 15:45:38.225937 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:38.225905 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" event={"ID":"f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4","Type":"ContainerStarted","Data":"fcc15fa5ced7c5e7021088a2425f5cf914934bb8896727a8ab2dcbe231bcc654"} Sep 08 15:45:38.993776 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:38.993745 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:45:39.074560 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074529 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bx7n\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-kube-api-access-4bx7n\") pod \"0e0de597-8fc8-48ce-b779-6f4936729811\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " Sep 08 15:45:39.074738 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074579 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-credential-socket\") pod \"0e0de597-8fc8-48ce-b779-6f4936729811\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " Sep 08 15:45:39.074738 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074630 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-envoy\") pod \"0e0de597-8fc8-48ce-b779-6f4936729811\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " Sep 08 15:45:39.074738 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074682 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-socket\") pod \"0e0de597-8fc8-48ce-b779-6f4936729811\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " Sep 08 15:45:39.074738 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074707 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-istio-token\") pod \"0e0de597-8fc8-48ce-b779-6f4936729811\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " Sep 08 15:45:39.074738 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074730 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0de597-8fc8-48ce-b779-6f4936729811-istio-podinfo\") pod \"0e0de597-8fc8-48ce-b779-6f4936729811\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " Sep 08 15:45:39.074977 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074763 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-data\") pod \"0e0de597-8fc8-48ce-b779-6f4936729811\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " Sep 08 15:45:39.074977 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074796 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0de597-8fc8-48ce-b779-6f4936729811-istiod-ca-cert\") pod \"0e0de597-8fc8-48ce-b779-6f4936729811\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " Sep 08 15:45:39.074977 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074867 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-certs\") pod \"0e0de597-8fc8-48ce-b779-6f4936729811\" (UID: \"0e0de597-8fc8-48ce-b779-6f4936729811\") " Sep 08 15:45:39.075155 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074981 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e0de597-8fc8-48ce-b779-6f4936729811" (UID: "0e0de597-8fc8-48ce-b779-6f4936729811"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:45:39.075155 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.074991 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e0de597-8fc8-48ce-b779-6f4936729811" (UID: "0e0de597-8fc8-48ce-b779-6f4936729811"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:45:39.075270 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.075212 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e0de597-8fc8-48ce-b779-6f4936729811" (UID: "0e0de597-8fc8-48ce-b779-6f4936729811"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:45:39.075270 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.075226 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e0de597-8fc8-48ce-b779-6f4936729811" (UID: "0e0de597-8fc8-48ce-b779-6f4936729811"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:45:39.075378 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.075307 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e0de597-8fc8-48ce-b779-6f4936729811-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e0de597-8fc8-48ce-b779-6f4936729811" (UID: "0e0de597-8fc8-48ce-b779-6f4936729811"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:45:39.075378 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.075324 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:39.075378 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.075345 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:39.075378 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.075361 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:39.075378 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.075375 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:39.077177 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.077148 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e0de597-8fc8-48ce-b779-6f4936729811" (UID: "0e0de597-8fc8-48ce-b779-6f4936729811"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:45:39.077300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.077175 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e0de597-8fc8-48ce-b779-6f4936729811" (UID: "0e0de597-8fc8-48ce-b779-6f4936729811"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:45:39.077300 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.077242 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-kube-api-access-4bx7n" (OuterVolumeSpecName: "kube-api-access-4bx7n") pod "0e0de597-8fc8-48ce-b779-6f4936729811" (UID: "0e0de597-8fc8-48ce-b779-6f4936729811"). InnerVolumeSpecName "kube-api-access-4bx7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:45:39.077414 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.077395 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e0de597-8fc8-48ce-b779-6f4936729811-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e0de597-8fc8-48ce-b779-6f4936729811" (UID: "0e0de597-8fc8-48ce-b779-6f4936729811"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:45:39.176216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.176068 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bx7n\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-kube-api-access-4bx7n\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:39.176216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.176103 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0de597-8fc8-48ce-b779-6f4936729811-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:39.176216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.176134 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0de597-8fc8-48ce-b779-6f4936729811-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:39.176216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.176146 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0de597-8fc8-48ce-b779-6f4936729811-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:39.176216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.176157 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0de597-8fc8-48ce-b779-6f4936729811-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:39.234932 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.234893 2578 generic.go:356] "Generic (PLEG): container finished" podID="0e0de597-8fc8-48ce-b779-6f4936729811" containerID="6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa" exitCode=0 Sep 08 15:45:39.235159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.234939 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" Sep 08 15:45:39.235159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.234980 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" event={"ID":"0e0de597-8fc8-48ce-b779-6f4936729811","Type":"ContainerDied","Data":"6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa"} Sep 08 15:45:39.235159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.235097 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2" event={"ID":"0e0de597-8fc8-48ce-b779-6f4936729811","Type":"ContainerDied","Data":"b6a671475645ac195c0e93dd689279ae90c79b06e7b815860cc4b78978981071"} Sep 08 15:45:39.235159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.235151 2578 scope.go:117] "RemoveContainer" containerID="6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa" Sep 08 15:45:39.236961 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.236926 2578 generic.go:356] "Generic (PLEG): container finished" podID="f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4" containerID="45fe879191093e0afd046630d7e19d12dff735b022cc7b8ac2aa286d4c895584" exitCode=0 Sep 08 15:45:39.237064 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.237011 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" event={"ID":"f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4","Type":"ContainerDied","Data":"45fe879191093e0afd046630d7e19d12dff735b022cc7b8ac2aa286d4c895584"} Sep 08 15:45:39.246408 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.246390 2578 scope.go:117] "RemoveContainer" containerID="a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb" Sep 08 15:45:39.258665 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.257653 2578 scope.go:117] "RemoveContainer" containerID="6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa" Sep 08 15:45:39.259092 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:45:39.258878 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa\": container with ID starting with 6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa not found: ID does not exist" containerID="6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa" Sep 08 15:45:39.259092 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.258924 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa"} err="failed to get container status \"6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa\": rpc error: code = NotFound desc = could not find container \"6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa\": container with ID starting with 6b730acc0b6c5658a3df77cb8edadb1e42725f952b690d07aa95e265a91414aa not found: ID does not exist" Sep 08 15:45:39.259092 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.258960 2578 scope.go:117] "RemoveContainer" containerID="a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb" Sep 08 15:45:39.259785 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:45:39.259744 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb\": container with ID starting with a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb not found: ID does not exist" containerID="a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb" Sep 08 15:45:39.259880 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.259790 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb"} err="failed to get container status \"a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb\": rpc error: code = NotFound desc = could not find container \"a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb\": container with ID starting with a043eddc919a9495c937b4c97d3d6f6cb8145c31a5f535223be3b437fb7217cb not found: ID does not exist" Sep 08 15:45:39.279949 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.279916 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2"] Sep 08 15:45:39.285665 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.285628 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-21c704-openshift-default-b6875fd74-sjlq2"] Sep 08 15:45:39.509059 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:39.509020 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" path="/var/lib/kubelet/pods/0e0de597-8fc8-48ce-b779-6f4936729811/volumes" Sep 08 15:45:40.387885 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:40.387858 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" Sep 08 15:45:40.490844 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:40.490802 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4-maas-api-service-ca\") pod \"f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4\" (UID: \"f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4\") " Sep 08 15:45:40.491218 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:40.491194 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4" (UID: "f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:45:40.592071 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:40.591963 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:45:41.253216 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:41.253185 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" Sep 08 15:45:41.253409 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:41.253225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q" event={"ID":"f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4","Type":"ContainerDied","Data":"fcc15fa5ced7c5e7021088a2425f5cf914934bb8896727a8ab2dcbe231bcc654"} Sep 08 15:45:41.253409 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:41.253279 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcc15fa5ced7c5e7021088a2425f5cf914934bb8896727a8ab2dcbe231bcc654" Sep 08 15:45:53.238301 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:53.238266 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8"] Sep 08 15:45:53.240952 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:53.240928 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-41de51-29814705-f77s8"] Sep 08 15:45:53.451389 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:53.451351 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q"] Sep 08 15:45:53.454044 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:53.454019 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41de51-c4j4q"] Sep 08 15:45:53.502965 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:53.502884 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69792bb7-1f80-4427-8490-0e73fd3a19e4" path="/var/lib/kubelet/pods/69792bb7-1f80-4427-8490-0e73fd3a19e4/volumes" Sep 08 15:45:53.503230 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:45:53.503216 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4" path="/var/lib/kubelet/pods/f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4/volumes" Sep 08 15:46:42.352104 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.352067 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw"] Sep 08 15:46:42.352653 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.352309 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" containerID="cri-o://a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb" gracePeriod=30 Sep 08 15:46:42.724395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724360 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x"] Sep 08 15:46:42.724831 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724816 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4" containerName="revoke-keys" Sep 08 15:46:42.724892 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724833 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4" containerName="revoke-keys" Sep 08 15:46:42.724892 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724847 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" Sep 08 15:46:42.724892 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724852 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" Sep 08 15:46:42.724892 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724870 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" Sep 08 15:46:42.724892 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724876 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" Sep 08 15:46:42.725175 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724961 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" Sep 08 15:46:42.725175 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724969 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e0de597-8fc8-48ce-b779-6f4936729811" containerName="istio-proxy" Sep 08 15:46:42.725175 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.724977 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1eea4d9-656b-4aaf-b8f1-e7e3d77510d4" containerName="revoke-keys" Sep 08 15:46:42.728325 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.728304 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" Sep 08 15:46:42.731088 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.731062 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:46:42.731258 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.731205 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:46:42.736642 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.736571 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x"] Sep 08 15:46:42.821169 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.821093 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6155d3ee-b031-4bb4-a158-ad817177f66d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x\" (UID: \"6155d3ee-b031-4bb4-a158-ad817177f66d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" Sep 08 15:46:42.922500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.922454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6155d3ee-b031-4bb4-a158-ad817177f66d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x\" (UID: \"6155d3ee-b031-4bb4-a158-ad817177f66d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" Sep 08 15:46:42.923070 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:42.923050 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6155d3ee-b031-4bb4-a158-ad817177f66d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x\" (UID: \"6155d3ee-b031-4bb4-a158-ad817177f66d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" Sep 08 15:46:43.040788 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:43.040679 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" Sep 08 15:46:43.325774 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:43.322060 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x"] Sep 08 15:46:43.674590 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:43.674547 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" event={"ID":"6155d3ee-b031-4bb4-a158-ad817177f66d","Type":"ContainerStarted","Data":"72ca135a1ecdaf208a185dc423b9d0531784ff0c47f2d82ea9260c1fe4a8c8f7"} Sep 08 15:46:43.674950 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:43.674613 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" event={"ID":"6155d3ee-b031-4bb4-a158-ad817177f66d","Type":"ContainerStarted","Data":"6b8ea7b4d879c83e9d2b0e6ba5faf06de548defd99d4019b9c0b0f580922db5b"} Sep 08 15:46:43.693123 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:43.693062 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" podStartSLOduration=1.6930477000000002 podStartE2EDuration="1.6930477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:46:43.691291461 +0000 UTC m=+1742.714915918" watchObservedRunningTime="2026-09-08 15:46:43.6930477 +0000 UTC m=+1742.716672158" Sep 08 15:46:44.684005 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:44.683967 2578 generic.go:356] "Generic (PLEG): container finished" podID="6155d3ee-b031-4bb4-a158-ad817177f66d" containerID="72ca135a1ecdaf208a185dc423b9d0531784ff0c47f2d82ea9260c1fe4a8c8f7" exitCode=0 Sep 08 15:46:44.684397 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:44.684011 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" event={"ID":"6155d3ee-b031-4bb4-a158-ad817177f66d","Type":"ContainerDied","Data":"72ca135a1ecdaf208a185dc423b9d0531784ff0c47f2d82ea9260c1fe4a8c8f7"} Sep 08 15:46:45.833914 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:45.833888 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" Sep 08 15:46:45.958050 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:45.958017 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6155d3ee-b031-4bb4-a158-ad817177f66d-maas-api-service-ca\") pod \"6155d3ee-b031-4bb4-a158-ad817177f66d\" (UID: \"6155d3ee-b031-4bb4-a158-ad817177f66d\") " Sep 08 15:46:45.958443 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:45.958416 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6155d3ee-b031-4bb4-a158-ad817177f66d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6155d3ee-b031-4bb4-a158-ad817177f66d" (UID: "6155d3ee-b031-4bb4-a158-ad817177f66d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:46:46.059606 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:46.059570 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6155d3ee-b031-4bb4-a158-ad817177f66d-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:46.697916 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:46.697885 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" Sep 08 15:46:46.698097 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:46.697923 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x" event={"ID":"6155d3ee-b031-4bb4-a158-ad817177f66d","Type":"ContainerDied","Data":"6b8ea7b4d879c83e9d2b0e6ba5faf06de548defd99d4019b9c0b0f580922db5b"} Sep 08 15:46:46.698097 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:46.697971 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b8ea7b4d879c83e9d2b0e6ba5faf06de548defd99d4019b9c0b0f580922db5b" Sep 08 15:46:47.606075 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.606048 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:46:47.674900 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.674811 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-certs\") pod \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " Sep 08 15:46:47.674900 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.674851 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2694v\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-kube-api-access-2694v\") pod \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " Sep 08 15:46:47.674900 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.674876 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-data\") pod \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " Sep 08 15:46:47.675228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.674928 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-credential-socket\") pod \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " Sep 08 15:46:47.675228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.674978 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-token\") pod \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " Sep 08 15:46:47.675228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675003 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istiod-ca-cert\") pod \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " Sep 08 15:46:47.675228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675025 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-socket\") pod \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " Sep 08 15:46:47.675228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675069 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-envoy\") pod \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " Sep 08 15:46:47.675228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675076 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fb9c44f8-661d-418c-88ad-0cf49eb075f8" (UID: "fb9c44f8-661d-418c-88ad-0cf49eb075f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:46:47.675228 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675143 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-podinfo\") pod \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\" (UID: \"fb9c44f8-661d-418c-88ad-0cf49eb075f8\") " Sep 08 15:46:47.675533 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675266 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fb9c44f8-661d-418c-88ad-0cf49eb075f8" (UID: "fb9c44f8-661d-418c-88ad-0cf49eb075f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:46:47.675533 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675283 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "fb9c44f8-661d-418c-88ad-0cf49eb075f8" (UID: "fb9c44f8-661d-418c-88ad-0cf49eb075f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:46:47.675533 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675341 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fb9c44f8-661d-418c-88ad-0cf49eb075f8" (UID: "fb9c44f8-661d-418c-88ad-0cf49eb075f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:46:47.675533 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675515 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:47.675533 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675529 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:47.675750 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675540 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:47.675750 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675549 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:47.675750 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.675677 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fb9c44f8-661d-418c-88ad-0cf49eb075f8" (UID: "fb9c44f8-661d-418c-88ad-0cf49eb075f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:46:47.677463 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.677439 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fb9c44f8-661d-418c-88ad-0cf49eb075f8" (UID: "fb9c44f8-661d-418c-88ad-0cf49eb075f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:46:47.677795 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.677775 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb9c44f8-661d-418c-88ad-0cf49eb075f8" (UID: "fb9c44f8-661d-418c-88ad-0cf49eb075f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:46:47.677869 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.677792 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-kube-api-access-2694v" (OuterVolumeSpecName: "kube-api-access-2694v") pod "fb9c44f8-661d-418c-88ad-0cf49eb075f8" (UID: "fb9c44f8-661d-418c-88ad-0cf49eb075f8"). InnerVolumeSpecName "kube-api-access-2694v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:46:47.677955 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.677935 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fb9c44f8-661d-418c-88ad-0cf49eb075f8" (UID: "fb9c44f8-661d-418c-88ad-0cf49eb075f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:46:47.707361 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.707328 2578 generic.go:356] "Generic (PLEG): container finished" podID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerID="a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb" exitCode=0 Sep 08 15:46:47.707561 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.707386 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" event={"ID":"fb9c44f8-661d-418c-88ad-0cf49eb075f8","Type":"ContainerDied","Data":"a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb"} Sep 08 15:46:47.707561 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.707436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" event={"ID":"fb9c44f8-661d-418c-88ad-0cf49eb075f8","Type":"ContainerDied","Data":"4388ce87421a9f03a329ddba5c0207238ecefce9d56ab0f468cd538928f37a62"} Sep 08 15:46:47.707561 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.707386 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw" Sep 08 15:46:47.707561 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.707466 2578 scope.go:117] "RemoveContainer" containerID="a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb" Sep 08 15:46:47.725396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.725369 2578 scope.go:117] "RemoveContainer" containerID="dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7" Sep 08 15:46:47.736330 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.736297 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw"] Sep 08 15:46:47.736616 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.736591 2578 scope.go:117] "RemoveContainer" containerID="a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb" Sep 08 15:46:47.737330 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:46:47.737288 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb\": container with ID starting with a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb not found: ID does not exist" containerID="a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb" Sep 08 15:46:47.737476 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.737342 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb"} err="failed to get container status \"a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb\": rpc error: code = NotFound desc = could not find container \"a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb\": container with ID starting with a94e0f0ebc98d080ed6cf1677bd593d258627c62a9665141545ba23bde2565bb not found: ID does not exist" Sep 08 15:46:47.737476 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.737370 2578 scope.go:117] "RemoveContainer" containerID="dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7" Sep 08 15:46:47.737744 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:46:47.737710 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7\": container with ID starting with dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7 not found: ID does not exist" containerID="dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7" Sep 08 15:46:47.737818 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.737764 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7"} err="failed to get container status \"dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7\": rpc error: code = NotFound desc = could not find container \"dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7\": container with ID starting with dd075544e323a332a0523dc97e3a96ba73748c0bc6b15c44800d281dfcd3edc7 not found: ID does not exist" Sep 08 15:46:47.741705 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.741678 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-41de51-openshift-default-64bb5f585d-2grcw"] Sep 08 15:46:47.776098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.776062 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:47.776098 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.776095 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:47.776338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.776139 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:47.776338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.776151 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb9c44f8-661d-418c-88ad-0cf49eb075f8-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:47.776338 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:47.776162 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2694v\" (UniqueName: \"kubernetes.io/projected/fb9c44f8-661d-418c-88ad-0cf49eb075f8-kube-api-access-2694v\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:46:49.502199 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:49.502157 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" path="/var/lib/kubelet/pods/fb9c44f8-661d-418c-88ad-0cf49eb075f8/volumes" Sep 08 15:46:53.257174 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:53.257138 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt"] Sep 08 15:46:53.260266 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:53.260240 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-0fea02-29814705-gjrzt"] Sep 08 15:46:53.484854 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:53.484801 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x"] Sep 08 15:46:53.487470 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:53.487441 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0fea02-nfn5x"] Sep 08 15:46:53.501982 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:53.501948 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b39e337-be08-4753-9c2a-b80c2eecb367" path="/var/lib/kubelet/pods/3b39e337-be08-4753-9c2a-b80c2eecb367/volumes" Sep 08 15:46:53.502307 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:46:53.502293 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6155d3ee-b031-4bb4-a158-ad817177f66d" path="/var/lib/kubelet/pods/6155d3ee-b031-4bb4-a158-ad817177f66d/volumes" Sep 08 15:47:00.599598 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.599562 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4"] Sep 08 15:47:00.600073 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.599799 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" containerID="cri-o://383bcb824586ebe48b988b3639efb7fdecf043c9ea8bca33195bb5355611bc1e" gracePeriod=30 Sep 08 15:47:00.972434 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.972394 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw"] Sep 08 15:47:00.972909 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.972892 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" Sep 08 15:47:00.972909 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.972909 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" Sep 08 15:47:00.973063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.972919 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6155d3ee-b031-4bb4-a158-ad817177f66d" containerName="revoke-keys" Sep 08 15:47:00.973063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.972926 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6155d3ee-b031-4bb4-a158-ad817177f66d" containerName="revoke-keys" Sep 08 15:47:00.973063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.973012 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" Sep 08 15:47:00.973063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.973021 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" Sep 08 15:47:00.973063 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.973030 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6155d3ee-b031-4bb4-a158-ad817177f66d" containerName="revoke-keys" Sep 08 15:47:00.977622 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.977601 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" Sep 08 15:47:00.980335 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.980309 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:47:00.980489 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.980352 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:47:00.985799 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:00.985654 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw"] Sep 08 15:47:01.009376 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:01.009338 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/674a9bcb-7227-4979-960f-d3f5ebdc182f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw\" (UID: \"674a9bcb-7227-4979-960f-d3f5ebdc182f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" Sep 08 15:47:01.110722 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:01.110687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/674a9bcb-7227-4979-960f-d3f5ebdc182f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw\" (UID: \"674a9bcb-7227-4979-960f-d3f5ebdc182f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" Sep 08 15:47:01.111407 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:01.111388 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/674a9bcb-7227-4979-960f-d3f5ebdc182f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw\" (UID: \"674a9bcb-7227-4979-960f-d3f5ebdc182f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" Sep 08 15:47:01.289467 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:01.289372 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" Sep 08 15:47:01.581404 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:01.581371 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw"] Sep 08 15:47:01.581525 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:47:01.581404 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod674a9bcb_7227_4979_960f_d3f5ebdc182f.slice/crio-67c076627ddfc53ac03ebeb46527bb91cc3232c45be2cde251122554126d76a3 WatchSource:0}: Error finding container 67c076627ddfc53ac03ebeb46527bb91cc3232c45be2cde251122554126d76a3: Status 404 returned error can't find the container with id 67c076627ddfc53ac03ebeb46527bb91cc3232c45be2cde251122554126d76a3 Sep 08 15:47:01.807746 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:01.807709 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" event={"ID":"674a9bcb-7227-4979-960f-d3f5ebdc182f","Type":"ContainerStarted","Data":"ff8fe0bbbde8dab67255eed5b6f42c4972c84a52611402fe1d77621e79441189"} Sep 08 15:47:01.808159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:01.807770 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" event={"ID":"674a9bcb-7227-4979-960f-d3f5ebdc182f","Type":"ContainerStarted","Data":"67c076627ddfc53ac03ebeb46527bb91cc3232c45be2cde251122554126d76a3"} Sep 08 15:47:01.829437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:01.829334 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" podStartSLOduration=1.829318283 podStartE2EDuration="1.829318283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:47:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:47:01.826955584 +0000 UTC m=+1760.850580041" watchObservedRunningTime="2026-09-08 15:47:01.829318283 +0000 UTC m=+1760.852942758" Sep 08 15:47:02.815804 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:02.815773 2578 generic.go:356] "Generic (PLEG): container finished" podID="674a9bcb-7227-4979-960f-d3f5ebdc182f" containerID="ff8fe0bbbde8dab67255eed5b6f42c4972c84a52611402fe1d77621e79441189" exitCode=0 Sep 08 15:47:02.816224 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:02.815866 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" event={"ID":"674a9bcb-7227-4979-960f-d3f5ebdc182f","Type":"ContainerDied","Data":"ff8fe0bbbde8dab67255eed5b6f42c4972c84a52611402fe1d77621e79441189"} Sep 08 15:47:03.964955 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:03.964927 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" Sep 08 15:47:04.039350 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:04.039309 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/674a9bcb-7227-4979-960f-d3f5ebdc182f-maas-api-service-ca\") pod \"674a9bcb-7227-4979-960f-d3f5ebdc182f\" (UID: \"674a9bcb-7227-4979-960f-d3f5ebdc182f\") " Sep 08 15:47:04.039687 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:04.039664 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/674a9bcb-7227-4979-960f-d3f5ebdc182f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "674a9bcb-7227-4979-960f-d3f5ebdc182f" (UID: "674a9bcb-7227-4979-960f-d3f5ebdc182f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:47:04.140493 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:04.140397 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/674a9bcb-7227-4979-960f-d3f5ebdc182f-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:04.837601 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:04.837573 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" Sep 08 15:47:04.837805 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:04.837565 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw" event={"ID":"674a9bcb-7227-4979-960f-d3f5ebdc182f","Type":"ContainerDied","Data":"67c076627ddfc53ac03ebeb46527bb91cc3232c45be2cde251122554126d76a3"} Sep 08 15:47:04.837805 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:04.837730 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67c076627ddfc53ac03ebeb46527bb91cc3232c45be2cde251122554126d76a3" Sep 08 15:47:05.336915 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.336876 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 15:47:05.337415 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.336958 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 15:47:05.847264 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.847235 2578 generic.go:356] "Generic (PLEG): container finished" podID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerID="383bcb824586ebe48b988b3639efb7fdecf043c9ea8bca33195bb5355611bc1e" exitCode=0 Sep 08 15:47:05.847422 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.847309 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" event={"ID":"2bc742a6-be92-4dac-a3b4-a311aa96a668","Type":"ContainerDied","Data":"383bcb824586ebe48b988b3639efb7fdecf043c9ea8bca33195bb5355611bc1e"} Sep 08 15:47:05.847422 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.847381 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" event={"ID":"2bc742a6-be92-4dac-a3b4-a311aa96a668","Type":"ContainerDied","Data":"24b88d08f3d1216b5458cf2dc37fda8b62e712538d1472fda211b8c46eea00e1"} Sep 08 15:47:05.847422 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.847407 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24b88d08f3d1216b5458cf2dc37fda8b62e712538d1472fda211b8c46eea00e1" Sep 08 15:47:05.847556 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.847426 2578 scope.go:117] "RemoveContainer" containerID="e12fab4c59617b5594f4471bb7a989ecf33716e03405d95446987ccd35ca33c6" Sep 08 15:47:05.851471 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.851449 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:47:05.956587 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956554 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-credential-socket\") pod \"2bc742a6-be92-4dac-a3b4-a311aa96a668\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " Sep 08 15:47:05.956779 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956596 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-envoy\") pod \"2bc742a6-be92-4dac-a3b4-a311aa96a668\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " Sep 08 15:47:05.956779 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956614 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-data\") pod \"2bc742a6-be92-4dac-a3b4-a311aa96a668\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " Sep 08 15:47:05.956779 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956653 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-podinfo\") pod \"2bc742a6-be92-4dac-a3b4-a311aa96a668\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " Sep 08 15:47:05.956779 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956673 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rv5bl\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-kube-api-access-rv5bl\") pod \"2bc742a6-be92-4dac-a3b4-a311aa96a668\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " Sep 08 15:47:05.956779 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956732 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-socket\") pod \"2bc742a6-be92-4dac-a3b4-a311aa96a668\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " Sep 08 15:47:05.956779 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956755 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-certs\") pod \"2bc742a6-be92-4dac-a3b4-a311aa96a668\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " Sep 08 15:47:05.957080 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956907 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-data" (OuterVolumeSpecName: "istio-data") pod "2bc742a6-be92-4dac-a3b4-a311aa96a668" (UID: "2bc742a6-be92-4dac-a3b4-a311aa96a668"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:05.957080 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956928 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-token\") pod \"2bc742a6-be92-4dac-a3b4-a311aa96a668\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " Sep 08 15:47:05.957080 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.956931 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2bc742a6-be92-4dac-a3b4-a311aa96a668" (UID: "2bc742a6-be92-4dac-a3b4-a311aa96a668"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:05.957080 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.957030 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc742a6-be92-4dac-a3b4-a311aa96a668-istiod-ca-cert\") pod \"2bc742a6-be92-4dac-a3b4-a311aa96a668\" (UID: \"2bc742a6-be92-4dac-a3b4-a311aa96a668\") " Sep 08 15:47:05.957335 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.957133 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2bc742a6-be92-4dac-a3b4-a311aa96a668" (UID: "2bc742a6-be92-4dac-a3b4-a311aa96a668"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:05.957396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.957360 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2bc742a6-be92-4dac-a3b4-a311aa96a668" (UID: "2bc742a6-be92-4dac-a3b4-a311aa96a668"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:05.957439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.957393 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bc742a6-be92-4dac-a3b4-a311aa96a668-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2bc742a6-be92-4dac-a3b4-a311aa96a668" (UID: "2bc742a6-be92-4dac-a3b4-a311aa96a668"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:47:05.957439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.957411 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:05.957439 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.957432 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:05.957540 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.957448 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:05.959211 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.959186 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2bc742a6-be92-4dac-a3b4-a311aa96a668" (UID: "2bc742a6-be92-4dac-a3b4-a311aa96a668"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:05.959323 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.959216 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-kube-api-access-rv5bl" (OuterVolumeSpecName: "kube-api-access-rv5bl") pod "2bc742a6-be92-4dac-a3b4-a311aa96a668" (UID: "2bc742a6-be92-4dac-a3b4-a311aa96a668"). InnerVolumeSpecName "kube-api-access-rv5bl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:47:05.959584 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.959562 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2bc742a6-be92-4dac-a3b4-a311aa96a668" (UID: "2bc742a6-be92-4dac-a3b4-a311aa96a668"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:47:05.959648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:05.959607 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-token" (OuterVolumeSpecName: "istio-token") pod "2bc742a6-be92-4dac-a3b4-a311aa96a668" (UID: "2bc742a6-be92-4dac-a3b4-a311aa96a668"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:47:06.058637 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:06.058594 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc742a6-be92-4dac-a3b4-a311aa96a668-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:06.058637 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:06.058626 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:06.058637 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:06.058635 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:06.058637 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:06.058645 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv5bl\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-kube-api-access-rv5bl\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:06.058905 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:06.058656 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc742a6-be92-4dac-a3b4-a311aa96a668-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:06.058905 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:06.058665 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc742a6-be92-4dac-a3b4-a311aa96a668-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:06.855552 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:06.855517 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4" Sep 08 15:47:06.906301 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:06.906257 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4"] Sep 08 15:47:06.911444 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:06.911402 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-0fea02-openshift-default-56db88bb5b-5bzw4"] Sep 08 15:47:07.502492 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:07.502454 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" path="/var/lib/kubelet/pods/2bc742a6-be92-4dac-a3b4-a311aa96a668/volumes" Sep 08 15:47:11.735297 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:11.735255 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6"] Sep 08 15:47:11.739849 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:11.739816 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-9ca1fc-29814705-qndg6"] Sep 08 15:47:12.000579 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:12.000496 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw"] Sep 08 15:47:12.002997 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:12.002971 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-9ca1fc-798nw"] Sep 08 15:47:13.501985 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:13.501948 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17be8490-19c1-4bda-b43e-ed5de68290ad" path="/var/lib/kubelet/pods/17be8490-19c1-4bda-b43e-ed5de68290ad/volumes" Sep 08 15:47:13.502477 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:13.502297 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="674a9bcb-7227-4979-960f-d3f5ebdc182f" path="/var/lib/kubelet/pods/674a9bcb-7227-4979-960f-d3f5ebdc182f/volumes" Sep 08 15:47:19.784101 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784061 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf"] Sep 08 15:47:19.784534 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784519 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" Sep 08 15:47:19.784534 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784535 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" Sep 08 15:47:19.784620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784545 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="674a9bcb-7227-4979-960f-d3f5ebdc182f" containerName="revoke-keys" Sep 08 15:47:19.784620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784550 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="674a9bcb-7227-4979-960f-d3f5ebdc182f" containerName="revoke-keys" Sep 08 15:47:19.784620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784560 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" Sep 08 15:47:19.784620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784567 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb9c44f8-661d-418c-88ad-0cf49eb075f8" containerName="istio-proxy" Sep 08 15:47:19.784620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784581 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" Sep 08 15:47:19.784620 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784585 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" Sep 08 15:47:19.784804 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784645 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" Sep 08 15:47:19.784804 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784654 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bc742a6-be92-4dac-a3b4-a311aa96a668" containerName="istio-proxy" Sep 08 15:47:19.784804 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.784663 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="674a9bcb-7227-4979-960f-d3f5ebdc182f" containerName="revoke-keys" Sep 08 15:47:19.788152 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.788134 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" Sep 08 15:47:19.791498 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.791464 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 15:47:19.791657 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.791470 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w9jgf\"" Sep 08 15:47:19.801036 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.801004 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf"] Sep 08 15:47:19.893825 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.893786 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd754ca-07e6-4223-bae7-51750ad727e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf\" (UID: \"9bd754ca-07e6-4223-bae7-51750ad727e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" Sep 08 15:47:19.994763 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.994725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd754ca-07e6-4223-bae7-51750ad727e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf\" (UID: \"9bd754ca-07e6-4223-bae7-51750ad727e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" Sep 08 15:47:19.995541 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:19.995517 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd754ca-07e6-4223-bae7-51750ad727e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf\" (UID: \"9bd754ca-07e6-4223-bae7-51750ad727e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" Sep 08 15:47:20.099934 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:20.099842 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" Sep 08 15:47:20.378909 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:20.378881 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf"] Sep 08 15:47:20.380900 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:47:20.380871 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bd754ca_07e6_4223_bae7_51750ad727e9.slice/crio-2aba02b88808ea328c0a90a7d0d1af349c6f6f844254922b644027c4af7b8600 WatchSource:0}: Error finding container 2aba02b88808ea328c0a90a7d0d1af349c6f6f844254922b644027c4af7b8600: Status 404 returned error can't find the container with id 2aba02b88808ea328c0a90a7d0d1af349c6f6f844254922b644027c4af7b8600 Sep 08 15:47:20.949455 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:20.949423 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" event={"ID":"9bd754ca-07e6-4223-bae7-51750ad727e9","Type":"ContainerStarted","Data":"979668a50aec71d373dbf66a6d0de1cbcd757ecab1941f1050803b6e7a4595bd"} Sep 08 15:47:20.949859 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:20.949476 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" event={"ID":"9bd754ca-07e6-4223-bae7-51750ad727e9","Type":"ContainerStarted","Data":"2aba02b88808ea328c0a90a7d0d1af349c6f6f844254922b644027c4af7b8600"} Sep 08 15:47:20.967656 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:20.967603 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" podStartSLOduration=1.967585669 podStartE2EDuration="1.967585669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:47:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:47:20.965997918 +0000 UTC m=+1779.989622373" watchObservedRunningTime="2026-09-08 15:47:20.967585669 +0000 UTC m=+1779.991210130" Sep 08 15:47:21.959068 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:21.959030 2578 generic.go:356] "Generic (PLEG): container finished" podID="9bd754ca-07e6-4223-bae7-51750ad727e9" containerID="979668a50aec71d373dbf66a6d0de1cbcd757ecab1941f1050803b6e7a4595bd" exitCode=0 Sep 08 15:47:21.959484 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:21.959088 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" event={"ID":"9bd754ca-07e6-4223-bae7-51750ad727e9","Type":"ContainerDied","Data":"979668a50aec71d373dbf66a6d0de1cbcd757ecab1941f1050803b6e7a4595bd"} Sep 08 15:47:23.101926 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:23.101899 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" Sep 08 15:47:23.230028 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:23.229928 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd754ca-07e6-4223-bae7-51750ad727e9-maas-api-service-ca\") pod \"9bd754ca-07e6-4223-bae7-51750ad727e9\" (UID: \"9bd754ca-07e6-4223-bae7-51750ad727e9\") " Sep 08 15:47:23.230396 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:23.230364 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bd754ca-07e6-4223-bae7-51750ad727e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9bd754ca-07e6-4223-bae7-51750ad727e9" (UID: "9bd754ca-07e6-4223-bae7-51750ad727e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:47:23.331066 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:23.331023 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd754ca-07e6-4223-bae7-51750ad727e9-maas-api-service-ca\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:23.973962 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:23.973930 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" Sep 08 15:47:23.974159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:23.973967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf" event={"ID":"9bd754ca-07e6-4223-bae7-51750ad727e9","Type":"ContainerDied","Data":"2aba02b88808ea328c0a90a7d0d1af349c6f6f844254922b644027c4af7b8600"} Sep 08 15:47:23.974159 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:23.974016 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2aba02b88808ea328c0a90a7d0d1af349c6f6f844254922b644027c4af7b8600" Sep 08 15:47:30.545215 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:30.545172 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525"] Sep 08 15:47:30.548794 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:30.548768 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9fb31f-29814705-hd525"] Sep 08 15:47:30.780286 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:30.780250 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf"] Sep 08 15:47:30.782758 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:30.782735 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9fb31f-gm9rf"] Sep 08 15:47:31.502575 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:31.502541 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c85a31a-0f29-48f7-ae85-95b24c9543b2" path="/var/lib/kubelet/pods/7c85a31a-0f29-48f7-ae85-95b24c9543b2/volumes" Sep 08 15:47:31.502846 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:31.502834 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bd754ca-07e6-4223-bae7-51750ad727e9" path="/var/lib/kubelet/pods/9bd754ca-07e6-4223-bae7-51750ad727e9/volumes" Sep 08 15:47:37.532369 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:37.532281 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r"] Sep 08 15:47:37.532789 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:37.532550 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" containerID="cri-o://0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36" gracePeriod=30 Sep 08 15:47:41.939771 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:41.939738 2578 scope.go:117] "RemoveContainer" containerID="0eba9643df6683e159223776555af5ca506d46bd5527507b647e97a01ec3bf85" Sep 08 15:47:41.950663 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:41.950633 2578 scope.go:117] "RemoveContainer" containerID="90c4cdfa8cb57dd4b9417bbbf3bea5fe4cf2a473915ed81b1e57766d11dd377f" Sep 08 15:47:42.799229 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.799165 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:47:42.924437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924398 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-envoy\") pod \"6a01f99c-66cf-45e5-808c-f24035420c28\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " Sep 08 15:47:42.924437 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924437 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-data\") pod \"6a01f99c-66cf-45e5-808c-f24035420c28\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " Sep 08 15:47:42.924680 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924480 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-socket\") pod \"6a01f99c-66cf-45e5-808c-f24035420c28\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " Sep 08 15:47:42.924680 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924496 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-credential-socket\") pod \"6a01f99c-66cf-45e5-808c-f24035420c28\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " Sep 08 15:47:42.924680 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924530 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-certs\") pod \"6a01f99c-66cf-45e5-808c-f24035420c28\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " Sep 08 15:47:42.924680 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924600 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a01f99c-66cf-45e5-808c-f24035420c28-istio-podinfo\") pod \"6a01f99c-66cf-45e5-808c-f24035420c28\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " Sep 08 15:47:42.924680 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924643 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a01f99c-66cf-45e5-808c-f24035420c28-istiod-ca-cert\") pod \"6a01f99c-66cf-45e5-808c-f24035420c28\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " Sep 08 15:47:42.924680 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924668 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlmtx\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-kube-api-access-jlmtx\") pod \"6a01f99c-66cf-45e5-808c-f24035420c28\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " Sep 08 15:47:42.924974 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924713 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-istio-token\") pod \"6a01f99c-66cf-45e5-808c-f24035420c28\" (UID: \"6a01f99c-66cf-45e5-808c-f24035420c28\") " Sep 08 15:47:42.924974 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924759 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a01f99c-66cf-45e5-808c-f24035420c28" (UID: "6a01f99c-66cf-45e5-808c-f24035420c28"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:42.924974 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924857 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a01f99c-66cf-45e5-808c-f24035420c28" (UID: "6a01f99c-66cf-45e5-808c-f24035420c28"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:42.924974 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924875 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a01f99c-66cf-45e5-808c-f24035420c28" (UID: "6a01f99c-66cf-45e5-808c-f24035420c28"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:42.925219 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.924964 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a01f99c-66cf-45e5-808c-f24035420c28" (UID: "6a01f99c-66cf-45e5-808c-f24035420c28"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:42.925219 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.925193 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:42.925219 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.925198 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a01f99c-66cf-45e5-808c-f24035420c28-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a01f99c-66cf-45e5-808c-f24035420c28" (UID: "6a01f99c-66cf-45e5-808c-f24035420c28"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 15:47:42.925219 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.925215 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-credential-socket\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:42.925421 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.925405 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-workload-certs\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:42.925476 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.925427 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-data\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:42.926932 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.926906 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a01f99c-66cf-45e5-808c-f24035420c28-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a01f99c-66cf-45e5-808c-f24035420c28" (UID: "6a01f99c-66cf-45e5-808c-f24035420c28"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 15:47:42.927034 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.926934 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a01f99c-66cf-45e5-808c-f24035420c28" (UID: "6a01f99c-66cf-45e5-808c-f24035420c28"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 15:47:42.927499 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.927481 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-kube-api-access-jlmtx" (OuterVolumeSpecName: "kube-api-access-jlmtx") pod "6a01f99c-66cf-45e5-808c-f24035420c28" (UID: "6a01f99c-66cf-45e5-808c-f24035420c28"). InnerVolumeSpecName "kube-api-access-jlmtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:47:42.927499 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:42.927491 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a01f99c-66cf-45e5-808c-f24035420c28" (UID: "6a01f99c-66cf-45e5-808c-f24035420c28"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 15:47:43.026191 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.026151 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a01f99c-66cf-45e5-808c-f24035420c28-istio-envoy\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:43.026191 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.026185 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a01f99c-66cf-45e5-808c-f24035420c28-istio-podinfo\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:43.026191 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.026193 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a01f99c-66cf-45e5-808c-f24035420c28-istiod-ca-cert\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:43.026191 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.026202 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlmtx\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-kube-api-access-jlmtx\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:43.026676 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.026211 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a01f99c-66cf-45e5-808c-f24035420c28-istio-token\") on node \"ip-10-0-132-25.ec2.internal\" DevicePath \"\"" Sep 08 15:47:43.097977 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.097877 2578 generic.go:356] "Generic (PLEG): container finished" podID="6a01f99c-66cf-45e5-808c-f24035420c28" containerID="0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36" exitCode=0 Sep 08 15:47:43.097977 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.097934 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" Sep 08 15:47:43.098215 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.097971 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" event={"ID":"6a01f99c-66cf-45e5-808c-f24035420c28","Type":"ContainerDied","Data":"0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36"} Sep 08 15:47:43.098215 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.098048 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r" event={"ID":"6a01f99c-66cf-45e5-808c-f24035420c28","Type":"ContainerDied","Data":"8518a3035336fadcb836c7f8628ddf86b9f90a4b473dc850a43f01f5837d94b3"} Sep 08 15:47:43.098215 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.098070 2578 scope.go:117] "RemoveContainer" containerID="0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36" Sep 08 15:47:43.108374 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.108355 2578 scope.go:117] "RemoveContainer" containerID="125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce" Sep 08 15:47:43.117962 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.117938 2578 scope.go:117] "RemoveContainer" containerID="0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36" Sep 08 15:47:43.118395 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:47:43.118358 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36\": container with ID starting with 0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36 not found: ID does not exist" containerID="0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36" Sep 08 15:47:43.118395 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.118408 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36"} err="failed to get container status \"0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36\": rpc error: code = NotFound desc = could not find container \"0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36\": container with ID starting with 0fd280c22778784d4d98a293d5b913c5d961522e03564739e7143e807d632b36 not found: ID does not exist" Sep 08 15:47:43.118691 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.118434 2578 scope.go:117] "RemoveContainer" containerID="125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce" Sep 08 15:47:43.118807 ip-10-0-132-25 kubenswrapper[2578]: E0908 15:47:43.118781 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce\": container with ID starting with 125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce not found: ID does not exist" containerID="125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce" Sep 08 15:47:43.118864 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.118817 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce"} err="failed to get container status \"125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce\": rpc error: code = NotFound desc = could not find container \"125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce\": container with ID starting with 125ba31631941e08963b6c42d55ce51bef065db0e434766725bf2b12163217ce not found: ID does not exist" Sep 08 15:47:43.121190 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.121166 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r"] Sep 08 15:47:43.125519 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.125495 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9fb31f-openshift-default-66789ff844-dmf4r"] Sep 08 15:47:43.502443 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:47:43.502409 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" path="/var/lib/kubelet/pods/6a01f99c-66cf-45e5-808c-f24035420c28/volumes" Sep 08 15:48:41.985123 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:48:41.985076 2578 scope.go:117] "RemoveContainer" containerID="e12cce63c36c2b90ce1e5ae61d5f10d2644a1f59d7e046a82df2a57ecdf8aa09" Sep 08 15:48:41.994564 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:48:41.994539 2578 scope.go:117] "RemoveContainer" containerID="2ae5b7f8ab771ec7aceebeb0dbf9f9bc3025f1f561b8303d6bd1446dce997784" Sep 08 15:50:42.039180 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:50:42.039080 2578 scope.go:117] "RemoveContainer" containerID="383bcb824586ebe48b988b3639efb7fdecf043c9ea8bca33195bb5355611bc1e" Sep 08 15:51:42.063393 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:51:42.063356 2578 scope.go:117] "RemoveContainer" containerID="45fe879191093e0afd046630d7e19d12dff735b022cc7b8ac2aa286d4c895584" Sep 08 15:51:42.073043 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:51:42.073020 2578 scope.go:117] "RemoveContainer" containerID="bdc3ae814e3fe80b367952bad345426ecd3857642ab64cfa5ff0f2a9169b70ba" Sep 08 15:51:42.082534 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:51:42.082508 2578 scope.go:117] "RemoveContainer" containerID="6cbb83c28d27ee2cf60e6987a8487b528d079f1c94bbaa8dbb7f907c37545465" Sep 08 15:51:42.091899 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:51:42.091879 2578 scope.go:117] "RemoveContainer" containerID="8a13d607cdb242b7730054529e60f19fb4c99c9b9047b4c8e5d5d25d597a581a" Sep 08 15:51:42.101223 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:51:42.101197 2578 scope.go:117] "RemoveContainer" containerID="4e9834436cfcf67dbc85d831495323e5b27f9eeb136e68a6a44b4ec32d903b9c" Sep 08 15:51:42.110021 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:51:42.109998 2578 scope.go:117] "RemoveContainer" containerID="e68034e30f4a73d3c161e0e8d004c35bdd39afae98780c930c0a439de406d12b" Sep 08 15:53:42.168577 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:53:42.168494 2578 scope.go:117] "RemoveContainer" containerID="979668a50aec71d373dbf66a6d0de1cbcd757ecab1941f1050803b6e7a4595bd" Sep 08 15:53:42.179198 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:53:42.179131 2578 scope.go:117] "RemoveContainer" containerID="72ca135a1ecdaf208a185dc423b9d0531784ff0c47f2d82ea9260c1fe4a8c8f7" Sep 08 15:53:42.188644 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:53:42.188623 2578 scope.go:117] "RemoveContainer" containerID="ff8fe0bbbde8dab67255eed5b6f42c4972c84a52611402fe1d77621e79441189" Sep 08 15:53:55.740863 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:53:55.740813 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-678545c99-wfv5g_7c456eae-1e57-45bd-95af-e546cfb0e52c/authorino/0.log" Sep 08 15:54:01.009385 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:01.009349 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pk6h9_01884b48-b914-4524-ba96-06c0840aac59/manager/0.log" Sep 08 15:54:01.361647 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:01.361573 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vfdjb_63fe5d59-d590-4aab-91c3-0883934916b5/server/0.log" Sep 08 15:54:01.710809 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:01.710779 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-847d47f6c8-fllhs_1be5ebad-b770-4552-aeac-66c2b640898e/copy-manifests/0.log" Sep 08 15:54:01.723309 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:01.723278 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-847d47f6c8-fllhs_1be5ebad-b770-4552-aeac-66c2b640898e/manager/0.log" Sep 08 15:54:03.458821 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:03.458784 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-678545c99-wfv5g_7c456eae-1e57-45bd-95af-e546cfb0e52c/authorino/0.log" Sep 08 15:54:03.583123 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:03.583074 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-g7vj7_264af2dd-1e4d-46dd-9467-b5d1d232903b/manager/0.log" Sep 08 15:54:03.692979 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:03.692948 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-4stlk_02326afe-568c-4990-9780-c3e68e1f47d1/manager/0.log" Sep 08 15:54:04.202545 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:04.202512 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-wc64b_71ecf2da-5125-400c-8b95-8de4eee2055d/manager/0.log" Sep 08 15:54:04.565076 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:04.564987 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j_d81c7232-d576-4049-9669-14e3e423fa43/istio-proxy/0.log" Sep 08 15:54:05.748052 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:05.748025 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-tbg75_42568245-ed3d-432d-824d-452457c0cc34/main/0.log" Sep 08 15:54:05.860648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:05.860617 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jclmz_efb86b57-5e55-4a1d-a9be-5f674fd80902/main/0.log" Sep 08 15:54:06.211040 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:06.211010 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv2zxw_03b4c9fc-e84a-4a41-b3a7-8057dd3c8dd0/main/0.log" Sep 08 15:54:11.853896 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.853859 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-czmnt/must-gather-76b27"] Sep 08 15:54:11.854318 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.854303 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bd754ca-07e6-4223-bae7-51750ad727e9" containerName="revoke-keys" Sep 08 15:54:11.854389 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.854319 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bd754ca-07e6-4223-bae7-51750ad727e9" containerName="revoke-keys" Sep 08 15:54:11.854389 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.854330 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" Sep 08 15:54:11.854389 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.854338 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" Sep 08 15:54:11.854389 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.854349 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" Sep 08 15:54:11.854389 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.854355 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" Sep 08 15:54:11.854669 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.854419 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" Sep 08 15:54:11.854669 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.854429 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bd754ca-07e6-4223-bae7-51750ad727e9" containerName="revoke-keys" Sep 08 15:54:11.854669 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.854436 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a01f99c-66cf-45e5-808c-f24035420c28" containerName="istio-proxy" Sep 08 15:54:11.857798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.857778 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czmnt/must-gather-76b27" Sep 08 15:54:11.863382 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.863339 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-czmnt\"/\"openshift-service-ca.crt\"" Sep 08 15:54:11.864798 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.864780 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-czmnt\"/\"default-dockercfg-fqwj2\"" Sep 08 15:54:11.865406 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.865385 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-czmnt\"/\"kube-root-ca.crt\"" Sep 08 15:54:11.888648 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.888604 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czmnt/must-gather-76b27"] Sep 08 15:54:11.922006 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.921970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45kgh\" (UniqueName: \"kubernetes.io/projected/cd552120-4ba2-4487-a017-36c2393b187b-kube-api-access-45kgh\") pod \"must-gather-76b27\" (UID: \"cd552120-4ba2-4487-a017-36c2393b187b\") " pod="openshift-must-gather-czmnt/must-gather-76b27" Sep 08 15:54:11.922220 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:11.922029 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cd552120-4ba2-4487-a017-36c2393b187b-must-gather-output\") pod \"must-gather-76b27\" (UID: \"cd552120-4ba2-4487-a017-36c2393b187b\") " pod="openshift-must-gather-czmnt/must-gather-76b27" Sep 08 15:54:12.023537 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:12.023498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45kgh\" (UniqueName: \"kubernetes.io/projected/cd552120-4ba2-4487-a017-36c2393b187b-kube-api-access-45kgh\") pod \"must-gather-76b27\" (UID: \"cd552120-4ba2-4487-a017-36c2393b187b\") " pod="openshift-must-gather-czmnt/must-gather-76b27" Sep 08 15:54:12.023723 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:12.023555 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cd552120-4ba2-4487-a017-36c2393b187b-must-gather-output\") pod \"must-gather-76b27\" (UID: \"cd552120-4ba2-4487-a017-36c2393b187b\") " pod="openshift-must-gather-czmnt/must-gather-76b27" Sep 08 15:54:12.023886 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:12.023868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cd552120-4ba2-4487-a017-36c2393b187b-must-gather-output\") pod \"must-gather-76b27\" (UID: \"cd552120-4ba2-4487-a017-36c2393b187b\") " pod="openshift-must-gather-czmnt/must-gather-76b27" Sep 08 15:54:12.038483 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:12.038450 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45kgh\" (UniqueName: \"kubernetes.io/projected/cd552120-4ba2-4487-a017-36c2393b187b-kube-api-access-45kgh\") pod \"must-gather-76b27\" (UID: \"cd552120-4ba2-4487-a017-36c2393b187b\") " pod="openshift-must-gather-czmnt/must-gather-76b27" Sep 08 15:54:12.167333 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:12.167235 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czmnt/must-gather-76b27" Sep 08 15:54:12.452343 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:12.452249 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czmnt/must-gather-76b27"] Sep 08 15:54:12.454814 ip-10-0-132-25 kubenswrapper[2578]: W0908 15:54:12.454786 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd552120_4ba2_4487_a017_36c2393b187b.slice/crio-c72f46bb3ad4f20545b5afbe732ebec4ca04edfab378b8041052758630376d45 WatchSource:0}: Error finding container c72f46bb3ad4f20545b5afbe732ebec4ca04edfab378b8041052758630376d45: Status 404 returned error can't find the container with id c72f46bb3ad4f20545b5afbe732ebec4ca04edfab378b8041052758630376d45 Sep 08 15:54:12.456685 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:12.456666 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 15:54:12.511971 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:12.511937 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czmnt/must-gather-76b27" event={"ID":"cd552120-4ba2-4487-a017-36c2393b187b","Type":"ContainerStarted","Data":"c72f46bb3ad4f20545b5afbe732ebec4ca04edfab378b8041052758630376d45"} Sep 08 15:54:14.527565 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:14.526885 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czmnt/must-gather-76b27" event={"ID":"cd552120-4ba2-4487-a017-36c2393b187b","Type":"ContainerStarted","Data":"5a281ff0ac945cc5b54c772ebb5a959c73d8a3e26ee75db38982dc25a5dd4db8"} Sep 08 15:54:14.527565 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:14.526937 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czmnt/must-gather-76b27" event={"ID":"cd552120-4ba2-4487-a017-36c2393b187b","Type":"ContainerStarted","Data":"b7497985fa09d5b46e9b741c040fc58796ce803d2e75ba19fd0f83597fbed145"} Sep 08 15:54:14.548502 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:14.548432 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-czmnt/must-gather-76b27" podStartSLOduration=2.559681435 podStartE2EDuration="3.548407603s" totalImagesPullingTime="988.726168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:54:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:54:14.546869506 +0000 UTC m=+2193.570493962" watchObservedRunningTime="2026-09-08 15:54:14.548407603 +0000 UTC m=+2193.572032063" Sep 08 15:54:15.088258 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:15.088222 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b6g8j_1baffed3-ce48-4637-abcd-01dd21feed17/global-pull-secret-syncer/0.log" Sep 08 15:54:15.262956 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:15.262914 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vt2b6_92a884a8-1be4-4cfa-bc2e-21dbcc43b79a/konnectivity-agent/0.log" Sep 08 15:54:15.289579 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:15.289526 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-25.ec2.internal_4d1459c4d11b8240fe56af12f6a1ae42/haproxy/0.log" Sep 08 15:54:20.646136 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:20.645587 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-678545c99-wfv5g_7c456eae-1e57-45bd-95af-e546cfb0e52c/authorino/0.log" Sep 08 15:54:20.681780 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:20.681475 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-g7vj7_264af2dd-1e4d-46dd-9467-b5d1d232903b/manager/0.log" Sep 08 15:54:20.715797 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:20.715742 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-4stlk_02326afe-568c-4990-9780-c3e68e1f47d1/manager/0.log" Sep 08 15:54:20.939918 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:20.939826 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-wc64b_71ecf2da-5125-400c-8b95-8de4eee2055d/manager/0.log" Sep 08 15:54:22.545281 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.545243 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_88f8d6ab-7ca6-429a-bdac-8579387ed04d/alertmanager/0.log" Sep 08 15:54:22.568931 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.568895 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_88f8d6ab-7ca6-429a-bdac-8579387ed04d/config-reloader/0.log" Sep 08 15:54:22.592959 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.592924 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_88f8d6ab-7ca6-429a-bdac-8579387ed04d/kube-rbac-proxy-web/0.log" Sep 08 15:54:22.615591 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.615557 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_88f8d6ab-7ca6-429a-bdac-8579387ed04d/kube-rbac-proxy/0.log" Sep 08 15:54:22.638977 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.638944 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_88f8d6ab-7ca6-429a-bdac-8579387ed04d/kube-rbac-proxy-metric/0.log" Sep 08 15:54:22.663876 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.663842 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_88f8d6ab-7ca6-429a-bdac-8579387ed04d/prom-label-proxy/0.log" Sep 08 15:54:22.688672 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.688599 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_88f8d6ab-7ca6-429a-bdac-8579387ed04d/init-config-reloader/0.log" Sep 08 15:54:22.758137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.758052 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-stxmz_21c79cef-1555-4a97-be07-4a52e9a4b965/kube-state-metrics/0.log" Sep 08 15:54:22.779128 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.779088 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-stxmz_21c79cef-1555-4a97-be07-4a52e9a4b965/kube-rbac-proxy-main/0.log" Sep 08 15:54:22.801592 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.801504 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-stxmz_21c79cef-1555-4a97-be07-4a52e9a4b965/kube-rbac-proxy-self/0.log" Sep 08 15:54:22.833365 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:22.833339 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6fdbc6dc49-qvnxp_c44cbd2f-5c56-4baa-84aa-9cee6ae7a859/metrics-server/0.log" Sep 08 15:54:23.143231 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.143140 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sw2zc_0c5dd6d3-221c-4795-90d4-9bc2526b7c8c/node-exporter/0.log" Sep 08 15:54:23.167579 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.167551 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sw2zc_0c5dd6d3-221c-4795-90d4-9bc2526b7c8c/kube-rbac-proxy/0.log" Sep 08 15:54:23.197498 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.197472 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sw2zc_0c5dd6d3-221c-4795-90d4-9bc2526b7c8c/init-textfile/0.log" Sep 08 15:54:23.223397 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.223370 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-2tr72_9a069451-f309-4a43-85c5-8a1844704331/kube-rbac-proxy-main/0.log" Sep 08 15:54:23.245501 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.245472 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-2tr72_9a069451-f309-4a43-85c5-8a1844704331/kube-rbac-proxy-self/0.log" Sep 08 15:54:23.269728 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.269692 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-2tr72_9a069451-f309-4a43-85c5-8a1844704331/openshift-state-metrics/0.log" Sep 08 15:54:23.504494 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.504459 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-jrdsd_34279f8e-45c5-4571-8055-baddd408eb55/prometheus-operator/0.log" Sep 08 15:54:23.525866 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.525830 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-jrdsd_34279f8e-45c5-4571-8055-baddd408eb55/kube-rbac-proxy/0.log" Sep 08 15:54:23.556171 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.556085 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-28xk5_b97f9898-ae7b-4bb1-b7b7-7a8b2cea6aaf/prometheus-operator-admission-webhook/0.log" Sep 08 15:54:23.591863 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.591833 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85b564cc54-jtt5r_aa349055-1cae-4668-a546-e6bcab9adabd/telemeter-client/0.log" Sep 08 15:54:23.613461 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.613431 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85b564cc54-jtt5r_aa349055-1cae-4668-a546-e6bcab9adabd/reload/0.log" Sep 08 15:54:23.638607 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.638575 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85b564cc54-jtt5r_aa349055-1cae-4668-a546-e6bcab9adabd/kube-rbac-proxy/0.log" Sep 08 15:54:23.679711 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.679677 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-794f578c9f-kcmjm_e042c070-b2f4-407a-adcb-28018c9572ed/thanos-query/0.log" Sep 08 15:54:23.703545 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.703519 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-794f578c9f-kcmjm_e042c070-b2f4-407a-adcb-28018c9572ed/kube-rbac-proxy-web/0.log" Sep 08 15:54:23.725261 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.725228 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-794f578c9f-kcmjm_e042c070-b2f4-407a-adcb-28018c9572ed/kube-rbac-proxy/0.log" Sep 08 15:54:23.754229 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.754182 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-794f578c9f-kcmjm_e042c070-b2f4-407a-adcb-28018c9572ed/prom-label-proxy/0.log" Sep 08 15:54:23.774313 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.774213 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-794f578c9f-kcmjm_e042c070-b2f4-407a-adcb-28018c9572ed/kube-rbac-proxy-rules/0.log" Sep 08 15:54:23.804371 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:23.804339 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-794f578c9f-kcmjm_e042c070-b2f4-407a-adcb-28018c9572ed/kube-rbac-proxy-metrics/0.log" Sep 08 15:54:25.596916 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.596874 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9"] Sep 08 15:54:25.602981 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.602942 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.608728 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.608682 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9"] Sep 08 15:54:25.676768 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.676724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-lib-modules\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.676969 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.676780 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-sys\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.676969 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.676839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-podres\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.676969 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.676863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j7hs\" (UniqueName: \"kubernetes.io/projected/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-kube-api-access-6j7hs\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.677170 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.677015 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-proc\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.778171 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.778126 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-proc\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.778381 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.778222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-lib-modules\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.778381 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.778252 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-sys\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.778381 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.778308 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-podres\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.778381 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.778330 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j7hs\" (UniqueName: \"kubernetes.io/projected/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-kube-api-access-6j7hs\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.778785 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.778759 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-proc\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.778913 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.778893 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-lib-modules\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.778979 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.778948 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-sys\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.779036 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.779019 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-podres\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.796853 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.796814 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j7hs\" (UniqueName: \"kubernetes.io/projected/cbf35a22-e19c-4a03-9b5b-fb14ece733e1-kube-api-access-6j7hs\") pod \"perf-node-gather-daemonset-jbdn9\" (UID: \"cbf35a22-e19c-4a03-9b5b-fb14ece733e1\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:25.924053 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:25.923948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:26.242137 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:26.241271 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9"] Sep 08 15:54:26.464325 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:26.464297 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66995fbfd7-zf9fn_9b4ddfc9-7e76-4dc3-960b-1905914dd893/console/0.log" Sep 08 15:54:26.498958 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:26.498862 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-cnffw_c3db4cab-73ab-45a2-9b43-ea0aa0bcb93c/download-server/0.log" Sep 08 15:54:26.645861 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:26.645829 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" event={"ID":"cbf35a22-e19c-4a03-9b5b-fb14ece733e1","Type":"ContainerStarted","Data":"187cc08565fb5695ef3d29479af14bf15e69e4c1636bda30091e3fe90d48ab78"} Sep 08 15:54:26.646320 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:26.645878 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" event={"ID":"cbf35a22-e19c-4a03-9b5b-fb14ece733e1","Type":"ContainerStarted","Data":"90e04607c9f67d1650837c7922b398a9b9a8f618873fc463ddca1cdd8aaab6bb"} Sep 08 15:54:26.646320 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:26.645974 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:26.663971 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:26.663914 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" podStartSLOduration=1.663897218 podStartE2EDuration="1.663897218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:54:26.662672258 +0000 UTC m=+2205.686296714" watchObservedRunningTime="2026-09-08 15:54:26.663897218 +0000 UTC m=+2205.687521673" Sep 08 15:54:28.156796 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:28.156766 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-crrxx_65e4ef42-8f74-405d-ba49-bef1f7c40c82/dns/0.log" Sep 08 15:54:28.176602 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:28.176572 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-crrxx_65e4ef42-8f74-405d-ba49-bef1f7c40c82/kube-rbac-proxy/0.log" Sep 08 15:54:28.324405 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:28.324377 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lx75p_0f26d665-ebd6-4939-861f-6085d394f06b/dns-node-resolver/0.log" Sep 08 15:54:28.856412 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:28.856381 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6866fd6c9c-fbltn_1b72473a-b0b1-4b67-913f-930bfffbbda0/registry/0.log" Sep 08 15:54:28.926539 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:28.926491 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zmlkq_efd21b96-4ff3-467f-89dd-f1ea27040971/node-ca/0.log" Sep 08 15:54:29.867074 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:29.867037 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j_d81c7232-d576-4049-9669-14e3e423fa43/istio-proxy/0.log" Sep 08 15:54:31.308196 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:31.308095 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nb2xs_ab614993-0c65-45df-9703-74ad6c70a1a4/serve-healthcheck-canary/0.log" Sep 08 15:54:32.009163 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:32.009133 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wsp27_204e0ebc-84af-4455-84d7-fb5fafbdf069/kube-rbac-proxy/0.log" Sep 08 15:54:32.031145 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:32.031100 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wsp27_204e0ebc-84af-4455-84d7-fb5fafbdf069/exporter/0.log" Sep 08 15:54:32.051631 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:32.051595 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wsp27_204e0ebc-84af-4455-84d7-fb5fafbdf069/extractor/0.log" Sep 08 15:54:32.667500 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:32.667466 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-jbdn9" Sep 08 15:54:34.478483 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:34.478452 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pk6h9_01884b48-b914-4524-ba96-06c0840aac59/manager/0.log" Sep 08 15:54:34.723053 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:34.723020 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vfdjb_63fe5d59-d590-4aab-91c3-0883934916b5/server/0.log" Sep 08 15:54:34.967246 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:34.967186 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-847d47f6c8-fllhs_1be5ebad-b770-4552-aeac-66c2b640898e/manager/0.log" Sep 08 15:54:34.986132 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:34.986084 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-847d47f6c8-fllhs_1be5ebad-b770-4552-aeac-66c2b640898e/copy-manifests/0.log" Sep 08 15:54:36.454317 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:36.454284 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bc9d6d9f9-4vg4f_9b080897-bb15-4de0-b1e1-bb96a0547cf3/manager/0.log" Sep 08 15:54:36.687232 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:36.687199 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j_d81c7232-d576-4049-9669-14e3e423fa43/istio-proxy/0.log" Sep 08 15:54:38.685737 ip-10-0-132-25 kubenswrapper[2578]: I0908 15:54:38.685702 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cc2p4j_d81c7232-d576-4049-9669-14e3e423fa43/istio-proxy/0.log"