Aug 04 20:41:05.729194 ip-10-0-139-54 systemd[1]: Starting Kubernetes Kubelet... Aug 04 20:41:06.161562 ip-10-0-139-54 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 20:41:06.161562 ip-10-0-139-54 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 20:41:06.161562 ip-10-0-139-54 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 20:41:06.161562 ip-10-0-139-54 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 20:41:06.161562 ip-10-0-139-54 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 20:41:06.163388 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.163292 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 20:41:06.167929 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167914 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 20:41:06.167929 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167928 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167932 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167935 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167938 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167941 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167946 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167949 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167952 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167955 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167958 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167961 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167964 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167966 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167969 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167971 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167975 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167979 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167982 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167985 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167988 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 20:41:06.167990 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167991 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167994 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.167996 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168000 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168002 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168005 2576 feature_gate.go:328] unrecognized feature gate: Example Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168008 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168010 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168012 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168015 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168018 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168020 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168023 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168025 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168027 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168030 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168032 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168035 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168037 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168042 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 20:41:06.168466 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168045 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168048 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168051 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168054 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168056 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168059 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168062 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168065 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168067 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168070 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168072 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168075 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168077 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168080 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168082 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168085 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168087 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168090 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168092 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 20:41:06.168979 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168095 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168097 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168100 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168102 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168105 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168107 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168110 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168112 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168114 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168117 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168119 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168123 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168125 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168128 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168130 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168133 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168136 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168138 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168140 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168144 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 20:41:06.169475 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168146 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168149 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168151 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168154 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168156 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168159 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168162 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168165 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168168 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168171 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168173 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168176 2576 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168179 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168181 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168184 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168187 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168191 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168194 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168197 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168890 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 20:41:06.169978 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168897 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168915 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168919 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168923 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168928 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168932 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168935 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168937 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168940 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168943 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168946 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168950 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168953 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168956 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168958 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168961 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168963 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168965 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168967 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168991 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 20:41:06.170449 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.168996 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169000 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169003 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169006 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169008 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169011 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169013 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169017 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169021 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169023 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169026 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169028 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169031 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169033 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169036 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169038 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169041 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169044 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169046 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 20:41:06.170959 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169049 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169051 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169053 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169056 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169059 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169062 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169064 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169067 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169070 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169073 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169075 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169078 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169080 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169083 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169086 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169088 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169091 2576 feature_gate.go:328] unrecognized feature gate: Example Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169093 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169096 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 20:41:06.171418 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169098 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169103 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169106 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169109 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169112 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169114 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169117 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169120 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169124 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169127 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169130 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169133 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169136 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169138 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169142 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169144 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169146 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169150 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169153 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 20:41:06.171879 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169155 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169158 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169160 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169163 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169165 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169168 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169170 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169173 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169175 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169178 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169193 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169196 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169198 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169201 2576 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169203 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169206 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169208 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169210 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169213 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169215 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 20:41:06.172357 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169218 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169572 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169577 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169579 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169582 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169585 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169588 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169591 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169593 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169596 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169600 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169602 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169605 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169607 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169610 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169612 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169615 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169617 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169619 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169622 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 20:41:06.172850 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169624 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169626 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169630 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169633 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169636 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169638 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169641 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169643 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169646 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169648 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169650 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169653 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169657 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169660 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169663 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169665 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169668 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169671 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169690 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 20:41:06.173348 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169694 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169697 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169699 2576 feature_gate.go:328] unrecognized feature gate: Example Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169703 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169708 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169711 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169714 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169717 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169720 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169723 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169726 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169728 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169731 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169733 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169735 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169738 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169740 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169743 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169745 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 20:41:06.173833 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169748 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169750 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169753 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169755 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169757 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169759 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169762 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169764 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169767 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169769 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169771 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169774 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169779 2576 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169781 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169785 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169787 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169790 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169793 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169795 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169798 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 20:41:06.174307 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169800 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169802 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169805 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169807 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169810 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169812 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169814 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169817 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169819 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169822 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169824 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169827 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169829 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169831 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169834 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169836 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169838 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169841 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169843 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169846 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 20:41:06.174793 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169848 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.169850 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169936 2576 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169943 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169952 2576 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169956 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169960 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169964 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169968 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169972 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169975 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169979 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169982 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169985 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169988 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169991 2576 flags.go:64] FLAG: --cgroup-root="" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169994 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.169997 2576 flags.go:64] FLAG: --client-ca-file="" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170000 2576 flags.go:64] FLAG: --cloud-provider="external" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170002 2576 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170006 2576 flags.go:64] FLAG: --cluster-domain="" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170009 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170012 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170015 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 20:41:06.175280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170018 2576 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170021 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170024 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170027 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170030 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170033 2576 flags.go:64] FLAG: --contention-profiling="false" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170035 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170039 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170042 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170045 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170049 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170052 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170055 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170058 2576 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170061 2576 flags.go:64] FLAG: --enable-server="true" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170064 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170069 2576 flags.go:64] FLAG: --event-burst="100" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170072 2576 flags.go:64] FLAG: --event-qps="50" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170075 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170078 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170080 2576 flags.go:64] FLAG: --eviction-hard="" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170084 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170087 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170090 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170093 2576 flags.go:64] FLAG: --eviction-soft="" Aug 04 20:41:06.175880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170095 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170098 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170101 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170104 2576 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170107 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170110 2576 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170113 2576 flags.go:64] FLAG: --feature-gates="" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170116 2576 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170120 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170123 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170126 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170129 2576 flags.go:64] FLAG: --healthz-port="10248" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170131 2576 flags.go:64] FLAG: --help="false" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170134 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170138 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170141 2576 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170144 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170147 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170150 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170153 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170156 2576 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170159 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170162 2576 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 20:41:06.176500 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170165 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170168 2576 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170171 2576 flags.go:64] FLAG: --kube-reserved="" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170174 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170176 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170179 2576 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170182 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170185 2576 flags.go:64] FLAG: --lock-file="" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170187 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170190 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170193 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170198 2576 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170201 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170203 2576 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170206 2576 flags.go:64] FLAG: --logging-format="text" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170209 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170212 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170215 2576 flags.go:64] FLAG: --manifest-url="" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170218 2576 flags.go:64] FLAG: --manifest-url-header="" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170222 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170225 2576 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170228 2576 flags.go:64] FLAG: --max-pods="110" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170231 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170235 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170238 2576 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 20:41:06.177110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170241 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170243 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170246 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170250 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170256 2576 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170259 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170262 2576 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170265 2576 flags.go:64] FLAG: --pod-cidr="" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170268 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170273 2576 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170276 2576 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170279 2576 flags.go:64] FLAG: --pods-per-core="0" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170282 2576 flags.go:64] FLAG: --port="10250" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170285 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170288 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0315d7499051cd501" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170291 2576 flags.go:64] FLAG: --qos-reserved="" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170294 2576 flags.go:64] FLAG: --read-only-port="10255" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170296 2576 flags.go:64] FLAG: --register-node="true" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170299 2576 flags.go:64] FLAG: --register-with-taints="" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170302 2576 flags.go:64] FLAG: --registry-burst="10" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170305 2576 flags.go:64] FLAG: --registry-qps="5" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170308 2576 flags.go:64] FLAG: --reserved-cpus="" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170311 2576 flags.go:64] FLAG: --reserved-memory="" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170314 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170317 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 20:41:06.177744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170320 2576 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170323 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170325 2576 flags.go:64] FLAG: --runonce="false" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170328 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170331 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170333 2576 flags.go:64] FLAG: --seccomp-default="false" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170337 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170340 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170343 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170346 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170349 2576 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170351 2576 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170354 2576 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170357 2576 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170360 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170362 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170366 2576 flags.go:64] FLAG: --system-cgroups="" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170368 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170373 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170376 2576 flags.go:64] FLAG: --tls-cert-file="" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170379 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170383 2576 flags.go:64] FLAG: --tls-min-version="" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170385 2576 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170388 2576 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170391 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 20:41:06.178404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170393 2576 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170396 2576 flags.go:64] FLAG: --v="2" Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170400 2576 flags.go:64] FLAG: --version="false" Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170403 2576 flags.go:64] FLAG: --vmodule="" Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170407 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.170410 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170522 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170525 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170528 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170530 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170533 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170536 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170538 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170543 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170545 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170548 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170551 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170554 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170557 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170559 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170562 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 20:41:06.179056 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170564 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170567 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170569 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170572 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170575 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170577 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170580 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170583 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170585 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170587 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170590 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170593 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170595 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170597 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170600 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170602 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170605 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170607 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170610 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170612 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 20:41:06.179566 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170616 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170620 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170623 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170626 2576 feature_gate.go:328] unrecognized feature gate: Example Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170630 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170632 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170635 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170638 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170642 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170644 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170647 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170649 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170652 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170655 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170657 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170660 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170663 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170665 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170668 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170670 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 20:41:06.180055 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170673 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170675 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170677 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170680 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170682 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170685 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170687 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170690 2576 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170692 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170695 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170697 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170700 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170702 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170705 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170707 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170710 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170713 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170716 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170719 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170721 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 20:41:06.180597 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170725 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170727 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170730 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170732 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170735 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170739 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170742 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170745 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170748 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170756 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170758 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170761 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170764 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170766 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170769 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170772 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170774 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170776 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170779 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 20:41:06.181111 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170782 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170784 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170788 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170791 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.170794 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.171478 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.179586 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.179599 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.180596 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.180604 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180671 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180676 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180680 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180683 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 20:41:06.181567 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180686 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180688 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180691 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180694 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180697 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180700 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180702 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180705 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180713 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180716 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180719 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180728 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180732 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180735 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180737 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180740 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180743 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180748 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180752 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 20:41:06.181938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180755 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180758 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180761 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180764 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180766 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180769 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180773 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180775 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180778 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180780 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180783 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180785 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180788 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180792 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180795 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180798 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180801 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180803 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180806 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 20:41:06.182399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180809 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180811 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180813 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180821 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180825 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180827 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180830 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180832 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180834 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180837 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180839 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180841 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180844 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180846 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180849 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180851 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180853 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180856 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180859 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180862 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 20:41:06.182889 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180865 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180867 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180870 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180873 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180875 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180877 2576 feature_gate.go:328] unrecognized feature gate: Example Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180880 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180883 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180886 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180888 2576 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180891 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180893 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180896 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180898 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180916 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180920 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180929 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180932 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180934 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180937 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 20:41:06.183389 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180939 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180941 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180944 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180946 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180949 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180951 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180953 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180956 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180958 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180961 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180963 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180967 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180969 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180972 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180975 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180978 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 20:41:06.183888 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.180980 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.180985 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181130 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181134 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181137 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181140 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181143 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181146 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181148 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181151 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181153 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181156 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181158 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181166 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181169 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 20:41:06.184351 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181171 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181174 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181176 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181179 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181181 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181183 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181186 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181188 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181191 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181193 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181196 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181198 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181202 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181204 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181207 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181209 2576 feature_gate.go:328] unrecognized feature gate: Example Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181212 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181214 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181217 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181219 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 20:41:06.184743 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181222 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181224 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181227 2576 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181229 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181232 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181234 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181237 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181239 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181243 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181247 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181249 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181258 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181261 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181264 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181266 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181269 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181272 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181275 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181277 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 20:41:06.185248 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181280 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181282 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181284 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181287 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181289 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181292 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181295 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181298 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181300 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181303 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181305 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181308 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181310 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181313 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181315 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181318 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181321 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181325 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181327 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 20:41:06.185707 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181330 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181332 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181335 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181343 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181346 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181348 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181358 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181360 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181363 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181365 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181368 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181370 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181373 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181375 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181377 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181380 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181382 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181385 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181387 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181389 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 20:41:06.186190 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181392 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181394 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181397 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181399 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181401 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181404 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181406 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:06.181408 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.181413 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.181517 2576 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.183872 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.185444 2576 server.go:1019] "Starting client certificate rotation" Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.185572 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 20:41:06.186703 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.186315 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 20:41:06.189981 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.189967 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 20:41:06.194228 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.191430 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 20:41:06.203102 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.203084 2576 log.go:25] "Validated CRI v1 runtime API" Aug 04 20:41:06.205285 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.205265 2576 log.go:25] "Validated CRI v1 image API" Aug 04 20:41:06.206133 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.206117 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 20:41:06.210817 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.210797 2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 dac3e1f4-945c-4bb2-9121-17b5fce968db:/dev/nvme0n1p3 e4e5f672-201b-452d-b9da-48b86465c1ee:/dev/nvme0n1p4] Aug 04 20:41:06.210888 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.210816 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 20:41:06.216006 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.215874 2576 manager.go:216] Machine: {Timestamp:2026-08-04 20:41:06.21538719 +0000 UTC m=+0.362257736 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094481 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2da748994dcb1f0c409fb2b61c26e3 SystemUUID:ec2da748-994d-cb1f-0c40-9fb2b61c26e3 BootID:17dbf787-37ed-49be-a2c9-c9cbd78151c1 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:27:76:e7:80:5d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:27:76:e7:80:5d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:ec:a3:22:6a:9a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 20:41:06.216006 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.216000 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 20:41:06.216137 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.216071 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 20:41:06.218516 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.218495 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 20:41:06.218644 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.218519 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-54.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 20:41:06.219327 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.219318 2576 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 20:41:06.219365 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.219330 2576 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 20:41:06.219365 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.219337 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 20:41:06.220166 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.220155 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 20:41:06.221206 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.221197 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 20:41:06.225296 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.225272 2576 state_mem.go:36] "Initialized new in-memory state store" Aug 04 20:41:06.225586 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.225573 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 20:41:06.225661 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.225614 2576 kubelet.go:486] "Attempting to sync node with API server" Aug 04 20:41:06.225661 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.225630 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 20:41:06.225661 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.225645 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 20:41:06.225661 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.225657 2576 kubelet.go:398] "Adding apiserver pod source" Aug 04 20:41:06.225824 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.225671 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 20:41:06.226078 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.226063 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 20:41:06.226159 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.226088 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 20:41:06.227013 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.226995 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 20:41:06.228350 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.228335 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 20:41:06.230302 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.230285 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 20:41:06.230384 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.230323 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 20:41:06.231996 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.231982 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 20:41:06.232066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232001 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 20:41:06.232066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232009 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 20:41:06.232066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232016 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 20:41:06.232066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232028 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 20:41:06.232066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232036 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 20:41:06.232066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232047 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 20:41:06.232066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232055 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 20:41:06.232066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232065 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 20:41:06.232301 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232074 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 20:41:06.232301 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232087 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 20:41:06.232301 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.232097 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 20:41:06.233047 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.233035 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 20:41:06.233107 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.233050 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 20:41:06.235810 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.235783 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-54.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 20:41:06.235896 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.235842 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 20:41:06.235896 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.235851 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 20:41:06.236281 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.236266 2576 server.go:1268] "Started kubelet" Aug 04 20:41:06.236410 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.236383 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 20:41:06.236410 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.236393 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 20:41:06.236551 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.236443 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 20:41:06.240993 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.240307 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 20:41:06.240642 ip-10-0-139-54 systemd[1]: Started Kubernetes Kubelet. Aug 04 20:41:06.242737 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.241882 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 20:41:06.245378 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.245363 2576 server.go:310] "Adding debug handlers to kubelet server" Aug 04 20:41:06.249160 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.249139 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 20:41:06.249160 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.249142 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 20:41:06.249297 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.249244 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:06.249859 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.249836 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 20:41:06.249969 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.249870 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 20:41:06.249969 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.249960 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 20:41:06.250075 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.249986 2576 reconstruct.go:97] "Volume reconstruction finished" Aug 04 20:41:06.250075 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.249996 2576 reconciler.go:29] "Reconciler: start to sync state" Aug 04 20:41:06.250075 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.250040 2576 factory.go:55] Registering systemd factory Aug 04 20:41:06.250075 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.250055 2576 factory.go:223] Registration of the systemd container factory successfully Aug 04 20:41:06.250286 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.250269 2576 factory.go:153] Registering CRI-O factory Aug 04 20:41:06.250286 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.250287 2576 factory.go:223] Registration of the crio container factory successfully Aug 04 20:41:06.250429 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.250336 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 20:41:06.250429 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.250361 2576 factory.go:103] Registering Raw factory Aug 04 20:41:06.250429 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.250375 2576 manager.go:1196] Started watching for new ooms in manager Aug 04 20:41:06.250757 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.250744 2576 manager.go:349] Starting recovery of all containers Aug 04 20:41:06.252950 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.252930 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mnwb4" Aug 04 20:41:06.259108 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.258953 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 04 20:41:06.259209 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.259176 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 04 20:41:06.260988 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.258954 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-54.ec2.internal.18c8b4d1fce20b28 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-54.ec2.internal,UID:ip-10-0-139-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-54.ec2.internal,},FirstTimestamp:2026-08-04 20:41:06.236238632 +0000 UTC m=+0.383109178,LastTimestamp:2026-08-04 20:41:06.236238632 +0000 UTC m=+0.383109178,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-54.ec2.internal,}" Aug 04 20:41:06.261233 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.261219 2576 manager.go:354] Recovery completed Aug 04 20:41:06.263360 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.263342 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mnwb4" Aug 04 20:41:06.265740 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.265724 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 20:41:06.268137 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.268121 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory" Aug 04 20:41:06.268208 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.268149 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 20:41:06.268208 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.268160 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID" Aug 04 20:41:06.268754 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.268739 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 20:41:06.268754 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.268751 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 20:41:06.268928 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.268754 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 20:41:06.268928 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.268794 2576 state_mem.go:36] "Initialized new in-memory state store" Aug 04 20:41:06.269526 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.269509 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory" Aug 04 20:41:06.269595 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.269538 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 20:41:06.269595 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.269547 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID" Aug 04 20:41:06.270055 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.270045 2576 policy_none.go:49] "None policy: Start" Aug 04 20:41:06.270108 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.270058 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 20:41:06.270108 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.270069 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 20:41:06.271270 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.271259 2576 policy_none.go:47] "Start" Aug 04 20:41:06.309445 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.309431 2576 manager.go:337] "Starting Device Plugin manager" Aug 04 20:41:06.309544 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.309465 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 20:41:06.309544 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.309474 2576 server.go:87] "Starting device plugin registration server" Aug 04 20:41:06.309696 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.309684 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 20:41:06.309736 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.309700 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 20:41:06.309797 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.309782 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 20:41:06.309893 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.309880 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 20:41:06.309969 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.309896 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 20:41:06.310462 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.310441 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 20:41:06.310517 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.310501 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:06.410309 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.410285 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 20:41:06.411491 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.411236 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory" Aug 04 20:41:06.411491 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.411266 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 20:41:06.411491 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.411277 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID" Aug 04 20:41:06.411491 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.411307 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.417925 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.417869 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.417925 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.417889 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-54.ec2.internal\": node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:06.432262 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.432237 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 20:41:06.433598 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.433585 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 20:41:06.433653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.433618 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 20:41:06.433653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.433636 2576 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 20:41:06.433738 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.433668 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 20:41:06.439199 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.439180 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:06.440186 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.440171 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 20:41:06.533991 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.533950 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal"] Aug 04 20:41:06.534127 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.534033 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 20:41:06.535048 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.535026 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory" Aug 04 20:41:06.535147 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.535071 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 20:41:06.535147 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.535085 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID" Aug 04 20:41:06.536602 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.536589 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 20:41:06.536732 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.536704 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.536769 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.536748 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 20:41:06.537350 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.537332 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory" Aug 04 20:41:06.537446 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.537364 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 20:41:06.537446 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.537374 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID" Aug 04 20:41:06.537446 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.537334 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory" Aug 04 20:41:06.537537 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.537447 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 20:41:06.537537 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.537459 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID" Aug 04 20:41:06.538677 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.538662 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.538751 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.538688 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 20:41:06.539280 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.539260 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:06.539392 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.539357 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory" Aug 04 20:41:06.539392 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.539389 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 20:41:06.539467 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.539405 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID" Aug 04 20:41:06.551790 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.551764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/87987e756c367b8b18190f03114b5471-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"87987e756c367b8b18190f03114b5471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.551873 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.551795 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87987e756c367b8b18190f03114b5471-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"87987e756c367b8b18190f03114b5471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.551873 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.551813 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/079dd266070347668559bc450b935f9a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-54.ec2.internal\" (UID: \"079dd266070347668559bc450b935f9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.563480 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.563448 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-54.ec2.internal\" not found" node="ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.567655 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.567639 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-54.ec2.internal\" not found" node="ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.640268 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.640245 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:06.652617 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.652593 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/079dd266070347668559bc450b935f9a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-54.ec2.internal\" (UID: \"079dd266070347668559bc450b935f9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.652707 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.652627 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/87987e756c367b8b18190f03114b5471-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"87987e756c367b8b18190f03114b5471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.652707 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.652650 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87987e756c367b8b18190f03114b5471-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"87987e756c367b8b18190f03114b5471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.652804 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.652704 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87987e756c367b8b18190f03114b5471-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"87987e756c367b8b18190f03114b5471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.652804 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.652706 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/87987e756c367b8b18190f03114b5471-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"87987e756c367b8b18190f03114b5471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.652804 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.652715 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/079dd266070347668559bc450b935f9a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-54.ec2.internal\" (UID: \"079dd266070347668559bc450b935f9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.740406 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.740341 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:06.840955 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.840922 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:06.866355 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.866335 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.870847 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:06.870695 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" Aug 04 20:41:06.941653 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:06.941622 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:07.042287 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.042216 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:07.142723 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.142688 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found" Aug 04 20:41:07.184190 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.184158 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 20:41:07.184784 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.184316 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 20:41:07.224152 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.224125 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 20:41:07.225944 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.225929 2576 apiserver.go:52] "Watching apiserver" Aug 04 20:41:07.236638 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.236616 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 20:41:07.236996 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.236976 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-55kxj","openshift-multus/multus-additional-cni-plugins-cbt9z","kube-system/global-pull-secret-syncer-6j5xp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc","openshift-cluster-node-tuning-operator/tuned-bqlbx","openshift-multus/multus-fdwtb","openshift-multus/network-metrics-daemon-qhmdn","openshift-network-diagnostics/network-check-target-lzgsl","openshift-network-operator/iptables-alerter-q7c5f","openshift-ovn-kubernetes/ovnkube-node-q5j4v","kube-system/konnectivity-agent-ncqlz","openshift-dns/node-resolver-4qgqw"] Aug 04 20:41:07.238616 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.238596 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.239966 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.239948 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.240938 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.240920 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l8xhf\"" Aug 04 20:41:07.241144 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.241129 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 20:41:07.241737 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.241494 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 20:41:07.241737 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.241530 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.241737 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.241637 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:07.241971 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.241858 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 20:41:07.242626 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.242607 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 20:41:07.242715 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.242630 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 20:41:07.242872 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.242855 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 20:41:07.243035 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.242941 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xv4lj\"" Aug 04 20:41:07.243104 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.243041 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 20:41:07.243990 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.243233 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 20:41:07.243990 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.243836 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.245177 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.245157 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.246037 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.246020 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hvqnw\"" Aug 04 20:41:07.246277 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.246260 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 20:41:07.246487 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.246473 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 20:41:07.246586 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.246573 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.246672 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.246655 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 20:41:07.247199 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.247181 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5dbpd\"" Aug 04 20:41:07.247279 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.247217 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 20:41:07.247336 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.247314 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 20:41:07.248047 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.248033 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:07.248105 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.248089 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:07.248423 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.248409 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 20:41:07.248989 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.248976 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6dnst\"" Aug 04 20:41:07.249290 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.249276 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 20:41:07.249355 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.249296 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" Aug 04 20:41:07.249579 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.249565 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:07.249641 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.249620 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:07.251065 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.251050 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.252673 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.252657 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.253393 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.253372 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 20:41:07.253393 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.253383 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 20:41:07.253541 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.253397 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9zz4j\"" Aug 04 20:41:07.253664 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.253651 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 20:41:07.254273 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.254260 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:07.255536 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.255517 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 20:41:07.255626 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.255518 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 20:41:07.255626 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.255610 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 20:41:07.255872 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.255855 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.256002 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.255982 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4fs62\"" Aug 04 20:41:07.256099 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256005 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 20:41:07.256099 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256012 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 20:41:07.256185 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256126 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 20:41:07.256268 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256236 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-socket-dir-parent\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.256373 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256281 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q4sn\" (UniqueName: \"kubernetes.io/projected/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-kube-api-access-5q4sn\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:07.256373 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256318 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd722b89-197e-48f9-b0a0-33e92b9eea90-iptables-alerter-script\") pod \"iptables-alerter-q7c5f\" (UID: \"dd722b89-197e-48f9-b0a0-33e92b9eea90\") " pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.256373 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-cni-dir\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.256373 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256368 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-cni-binary-copy\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.256536 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256390 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46r2k\" (UniqueName: \"kubernetes.io/projected/4004db00-a176-456f-88d5-a8204195d9d3-kube-api-access-46r2k\") pod \"node-ca-55kxj\" (UID: \"4004db00-a176-456f-88d5-a8204195d9d3\") " pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.256536 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256409 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-modprobe-d\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.256536 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256431 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-sys\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.256536 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256451 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-host\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.256536 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256500 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-tmp\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.256536 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256523 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-system-cni-dir\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256540 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256547 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84nj6\" (UniqueName: \"kubernetes.io/projected/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-kube-api-access-84nj6\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256603 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-run-k8s-cni-cncf-io\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256628 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tchj\" (UniqueName: \"kubernetes.io/projected/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-kube-api-access-8tchj\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256657 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-khxfb\"" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256677 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256664 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-etc-kubernetes\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-run\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256790 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.256812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256812 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-device-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256828 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-sysctl-conf\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256842 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-run-multus-certs\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.256878 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257048 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257076 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-var-lib-cni-multus\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257111 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257135 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-sysconfig\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257168 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-var-lib-cni-bin\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257231 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-var-lib-kubelet\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257256 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-systemd\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.257287 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257278 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-daemon-config\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257314 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257360 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a43ca81a-b9a5-45da-b3fa-09452b3296f6-kubelet-config\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257388 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-socket-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257422 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-var-lib-kubelet\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257450 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-cnibin\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257489 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-cnibin\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257514 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfdhf\" (UniqueName: \"kubernetes.io/projected/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-kube-api-access-lfdhf\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257539 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a43ca81a-b9a5-45da-b3fa-09452b3296f6-dbus\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257566 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-lib-modules\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl6l8\" (UniqueName: \"kubernetes.io/projected/dd722b89-197e-48f9-b0a0-33e92b9eea90-kube-api-access-xl6l8\") pod \"iptables-alerter-q7c5f\" (UID: \"dd722b89-197e-48f9-b0a0-33e92b9eea90\") " pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257625 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-run-netns\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257648 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-hostroot\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257680 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-system-cni-dir\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257721 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-os-release\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.257742 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257747 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-registration-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257778 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257807 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-sysctl-d\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257837 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk2pq\" (UniqueName: \"kubernetes.io/projected/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-kube-api-access-nk2pq\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257860 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-os-release\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257883 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-conf-dir\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257942 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-tuned\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.257992 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd722b89-197e-48f9-b0a0-33e92b9eea90-host-slash\") pod \"iptables-alerter-q7c5f\" (UID: \"dd722b89-197e-48f9-b0a0-33e92b9eea90\") " pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258021 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4004db00-a176-456f-88d5-a8204195d9d3-host\") pod \"node-ca-55kxj\" (UID: \"4004db00-a176-456f-88d5-a8204195d9d3\") " pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-sys-fs\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258152 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4004db00-a176-456f-88d5-a8204195d9d3-serviceca\") pod \"node-ca-55kxj\" (UID: \"4004db00-a176-456f-88d5-a8204195d9d3\") " pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258175 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-cni-binary-copy\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258261 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-kubernetes\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258533 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hwk4g\"" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258640 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 20:41:07.258891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.258744 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 20:41:07.266425 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.266410 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 20:41:07.266497 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.266487 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" Aug 04 20:41:07.267131 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.267112 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"] Aug 04 20:41:07.267173 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.267131 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 20:36:06 +0000 UTC" deadline="2028-02-22 17:28:46.681429789 +0000 UTC" Aug 04 20:41:07.267205 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.267174 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13604h47m39.41425796s" Aug 04 20:41:07.269443 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.269424 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 20:41:07.275551 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.275534 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 20:41:07.275691 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.275677 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal"] Aug 04 20:41:07.291485 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.291468 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4jqn" Aug 04 20:41:07.300666 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.300620 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4jqn" Aug 04 20:41:07.351095 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.351066 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 20:41:07.359410 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359386 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.359538 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359415 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-device-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.359538 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359437 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-sysctl-conf\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.359538 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359453 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-run-multus-certs\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.359538 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359481 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/be9e984d-1b26-4f1c-8403-aa0790833522-ovn-node-metrics-cert\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.359538 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359506 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.359538 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359529 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359535 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-device-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359543 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-run-multus-certs\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359551 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-var-lib-cni-multus\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359592 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-var-lib-cni-multus\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359597 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-sysctl-conf\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359604 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-run-netns\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359640 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/be9e984d-1b26-4f1c-8403-aa0790833522-ovnkube-script-lib\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359696 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-sysconfig\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359725 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-var-lib-cni-bin\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359749 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-var-lib-kubelet\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359791 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-systemd\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359818 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-run-openvswitch\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.359842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359846 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-daemon-config\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359877 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359890 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-var-lib-kubelet\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359904 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a43ca81a-b9a5-45da-b3fa-09452b3296f6-kubelet-config\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359968 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-systemd\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359991 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-sysconfig\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.359993 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-var-lib-cni-bin\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.360009 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360071 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a43ca81a-b9a5-45da-b3fa-09452b3296f6-kubelet-config\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.360209 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret podName:a43ca81a-b9a5-45da-b3fa-09452b3296f6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:07.860160952 +0000 UTC m=+2.007031503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret") pod "global-pull-secret-syncer-6j5xp" (UID: "a43ca81a-b9a5-45da-b3fa-09452b3296f6") : object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360292 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360444 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-socket-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360606 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360628 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-var-lib-kubelet\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360655 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-cnibin\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360692 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-log-socket\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.361465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360697 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-socket-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360734 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-cni-bin\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360759 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-var-lib-kubelet\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360769 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-cnibin\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360764 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-cnibin\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360794 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-cnibin\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360819 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfdhf\" (UniqueName: \"kubernetes.io/projected/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-kube-api-access-lfdhf\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360829 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-daemon-config\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360845 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a43ca81a-b9a5-45da-b3fa-09452b3296f6-dbus\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-lib-modules\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360937 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl6l8\" (UniqueName: \"kubernetes.io/projected/dd722b89-197e-48f9-b0a0-33e92b9eea90-kube-api-access-xl6l8\") pod \"iptables-alerter-q7c5f\" (UID: \"dd722b89-197e-48f9-b0a0-33e92b9eea90\") " pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360952 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a43ca81a-b9a5-45da-b3fa-09452b3296f6-dbus\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.360971 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-slash\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361021 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-etc-openvswitch\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361025 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-lib-modules\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361045 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92qzh\" (UniqueName: \"kubernetes.io/projected/be9e984d-1b26-4f1c-8403-aa0790833522-kube-api-access-92qzh\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361074 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-run-netns\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.362389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361114 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-run-netns\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361136 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-hostroot\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361163 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-system-cni-dir\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361189 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-os-release\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361215 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-registration-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361240 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361244 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-hostroot\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361265 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-sysctl-d\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk2pq\" (UniqueName: \"kubernetes.io/projected/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-kube-api-access-nk2pq\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361333 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-os-release\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361354 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-conf-dir\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361376 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-tuned\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361398 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd722b89-197e-48f9-b0a0-33e92b9eea90-host-slash\") pod \"iptables-alerter-q7c5f\" (UID: \"dd722b89-197e-48f9-b0a0-33e92b9eea90\") " pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361242 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-system-cni-dir\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361415 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-sysctl-d\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361306 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-os-release\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.363406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361463 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/be9e984d-1b26-4f1c-8403-aa0790833522-env-overrides\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361488 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-conf-dir\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361500 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-os-release\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361509 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eef28aa9-8bf5-4c87-bcb6-245360c85048-konnectivity-ca\") pod \"konnectivity-agent-ncqlz\" (UID: \"eef28aa9-8bf5-4c87-bcb6-245360c85048\") " pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361353 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361537 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4004db00-a176-456f-88d5-a8204195d9d3-host\") pod \"node-ca-55kxj\" (UID: \"4004db00-a176-456f-88d5-a8204195d9d3\") " pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361566 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd722b89-197e-48f9-b0a0-33e92b9eea90-host-slash\") pod \"iptables-alerter-q7c5f\" (UID: \"dd722b89-197e-48f9-b0a0-33e92b9eea90\") " pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361565 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361610 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-sys-fs\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361619 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361638 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-cni-netd\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361665 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361693 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/157ecd5c-7c37-4d11-8ec9-947628a5540d-tmp-dir\") pod \"node-resolver-4qgqw\" (UID: \"157ecd5c-7c37-4d11-8ec9-947628a5540d\") " pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361746 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-var-lib-openvswitch\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361774 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361771 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/be9e984d-1b26-4f1c-8403-aa0790833522-ovnkube-config\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.364135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361794 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4004db00-a176-456f-88d5-a8204195d9d3-host\") pod \"node-ca-55kxj\" (UID: \"4004db00-a176-456f-88d5-a8204195d9d3\") " pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361835 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4004db00-a176-456f-88d5-a8204195d9d3-serviceca\") pod \"node-ca-55kxj\" (UID: \"4004db00-a176-456f-88d5-a8204195d9d3\") " pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361866 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-sys-fs\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-cni-binary-copy\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361905 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-kubernetes\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361937 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-socket-dir-parent\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361995 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q4sn\" (UniqueName: \"kubernetes.io/projected/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-kube-api-access-5q4sn\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362025 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-run-ovn\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362053 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd722b89-197e-48f9-b0a0-33e92b9eea90-iptables-alerter-script\") pod \"iptables-alerter-q7c5f\" (UID: \"dd722b89-197e-48f9-b0a0-33e92b9eea90\") " pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362079 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-cni-dir\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362097 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-cni-binary-copy\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/157ecd5c-7c37-4d11-8ec9-947628a5540d-hosts-file\") pod \"node-resolver-4qgqw\" (UID: \"157ecd5c-7c37-4d11-8ec9-947628a5540d\") " pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362144 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46r2k\" (UniqueName: \"kubernetes.io/projected/4004db00-a176-456f-88d5-a8204195d9d3-kube-api-access-46r2k\") pod \"node-ca-55kxj\" (UID: \"4004db00-a176-456f-88d5-a8204195d9d3\") " pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362169 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-modprobe-d\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-sys\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-host\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-tmp\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.364775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362254 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-system-cni-dir\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362273 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84nj6\" (UniqueName: \"kubernetes.io/projected/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-kube-api-access-84nj6\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362290 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-kubelet\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362319 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-run-k8s-cni-cncf-io\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.361398 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-registration-dir\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362365 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tchj\" (UniqueName: \"kubernetes.io/projected/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-kube-api-access-8tchj\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362382 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-etc-kubernetes\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362405 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-systemd-units\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362425 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-run-systemd\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362412 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-cni-dir\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362444 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-node-log\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362415 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-cni-binary-copy\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362493 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eef28aa9-8bf5-4c87-bcb6-245360c85048-agent-certs\") pod \"konnectivity-agent-ncqlz\" (UID: \"eef28aa9-8bf5-4c87-bcb6-245360c85048\") " pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362532 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgjvg\" (UniqueName: \"kubernetes.io/projected/157ecd5c-7c37-4d11-8ec9-947628a5540d-kube-api-access-fgjvg\") pod \"node-resolver-4qgqw\" (UID: \"157ecd5c-7c37-4d11-8ec9-947628a5540d\") " pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362617 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-run\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362831 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-etc-kubernetes\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362871 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-run\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.365462 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362952 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-multus-socket-dir-parent\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.362967 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4004db00-a176-456f-88d5-a8204195d9d3-serviceca\") pod \"node-ca-55kxj\" (UID: \"4004db00-a176-456f-88d5-a8204195d9d3\") " pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.363070 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-system-cni-dir\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.363103 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.363122 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-cni-binary-copy\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.363169 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-modprobe-d\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.363238 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-sys\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.363283 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs podName:e6116bd3-50d9-4d61-95c1-ea0cc53b88ab nodeName:}" failed. No retries permitted until 2026-08-04 20:41:07.863258966 +0000 UTC m=+2.010129523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs") pod "network-metrics-daemon-qhmdn" (UID: "e6116bd3-50d9-4d61-95c1-ea0cc53b88ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.363291 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-host\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.363363 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-host-run-k8s-cni-cncf-io\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.363692 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd722b89-197e-48f9-b0a0-33e92b9eea90-iptables-alerter-script\") pod \"iptables-alerter-q7c5f\" (UID: \"dd722b89-197e-48f9-b0a0-33e92b9eea90\") " pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.363719 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-kubernetes\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.365502 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-etc-tuned\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.366097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.365897 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-tmp\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.367811 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.367796 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 20:41:07.367866 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.367814 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 20:41:07.367866 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.367824 2576 projected.go:196] Error preparing data for projected volume kube-api-access-d8wn6 for pod openshift-network-diagnostics/network-check-target-lzgsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:07.367943 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.367881 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6 podName:47981c6e-01ce-496b-8d23-7c1c795cdbc6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:07.867870002 +0000 UTC m=+2.014740538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d8wn6" (UniqueName: "kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6") pod "network-check-target-lzgsl" (UID: "47981c6e-01ce-496b-8d23-7c1c795cdbc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:07.371432 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.371395 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl6l8\" (UniqueName: \"kubernetes.io/projected/dd722b89-197e-48f9-b0a0-33e92b9eea90-kube-api-access-xl6l8\") pod \"iptables-alerter-q7c5f\" (UID: \"dd722b89-197e-48f9-b0a0-33e92b9eea90\") " pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.371567 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.371540 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfdhf\" (UniqueName: \"kubernetes.io/projected/0aa9cd51-695a-4ad9-b14b-5fe0dc758268-kube-api-access-lfdhf\") pod \"multus-additional-cni-plugins-cbt9z\" (UID: \"0aa9cd51-695a-4ad9-b14b-5fe0dc758268\") " pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.372118 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.372089 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46r2k\" (UniqueName: \"kubernetes.io/projected/4004db00-a176-456f-88d5-a8204195d9d3-kube-api-access-46r2k\") pod \"node-ca-55kxj\" (UID: \"4004db00-a176-456f-88d5-a8204195d9d3\") " pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.372223 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.372208 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk2pq\" (UniqueName: \"kubernetes.io/projected/f0a71e92-5c9b-470f-83b8-ae4c98ae7a34-kube-api-access-nk2pq\") pod \"tuned-bqlbx\" (UID: \"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34\") " pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.381579 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.381355 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tchj\" (UniqueName: \"kubernetes.io/projected/a20f48cb-c3f8-4a84-8423-fcd9da9ad156-kube-api-access-8tchj\") pod \"aws-ebs-csi-driver-node-fpczc\" (UID: \"a20f48cb-c3f8-4a84-8423-fcd9da9ad156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.382693 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.382669 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q4sn\" (UniqueName: \"kubernetes.io/projected/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-kube-api-access-5q4sn\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:07.382796 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.382752 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84nj6\" (UniqueName: \"kubernetes.io/projected/1c8bcaf1-8188-447a-84e0-06e13d2cadb2-kube-api-access-84nj6\") pod \"multus-fdwtb\" (UID: \"1c8bcaf1-8188-447a-84e0-06e13d2cadb2\") " pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.443197 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.443170 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87987e756c367b8b18190f03114b5471.slice/crio-c9194213adf630159611a962eb3743397491e454a2a7e8d03de9e8e246d20c43 WatchSource:0}: Error finding container c9194213adf630159611a962eb3743397491e454a2a7e8d03de9e8e246d20c43: Status 404 returned error can't find the container with id c9194213adf630159611a962eb3743397491e454a2a7e8d03de9e8e246d20c43 Aug 04 20:41:07.443435 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.443415 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079dd266070347668559bc450b935f9a.slice/crio-0adc6343dd48f9dffe64d4e1125d7b52af8207295b7cc3ffb6552a626013be98 WatchSource:0}: Error finding container 0adc6343dd48f9dffe64d4e1125d7b52af8207295b7cc3ffb6552a626013be98: Status 404 returned error can't find the container with id 0adc6343dd48f9dffe64d4e1125d7b52af8207295b7cc3ffb6552a626013be98 Aug 04 20:41:07.450377 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.450129 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 20:41:07.464913 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.464874 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-cni-netd\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465011 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.464913 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465011 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.464961 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/157ecd5c-7c37-4d11-8ec9-947628a5540d-tmp-dir\") pod \"node-resolver-4qgqw\" (UID: \"157ecd5c-7c37-4d11-8ec9-947628a5540d\") " pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.465011 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.464992 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-var-lib-openvswitch\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465010 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/be9e984d-1b26-4f1c-8403-aa0790833522-ovnkube-config\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-run-ovn\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465061 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/157ecd5c-7c37-4d11-8ec9-947628a5540d-hosts-file\") pod \"node-resolver-4qgqw\" (UID: \"157ecd5c-7c37-4d11-8ec9-947628a5540d\") " pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465062 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465089 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-kubelet\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465078 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-cni-netd\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465105 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-var-lib-openvswitch\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465106 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-run-ovn\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465119 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-systemd-units\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465153 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/157ecd5c-7c37-4d11-8ec9-947628a5540d-hosts-file\") pod \"node-resolver-4qgqw\" (UID: \"157ecd5c-7c37-4d11-8ec9-947628a5540d\") " pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465161 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-kubelet\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465162 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-systemd-units\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465175 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-run-systemd\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465201 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-node-log\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465210 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-run-systemd\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465226 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eef28aa9-8bf5-4c87-bcb6-245360c85048-agent-certs\") pod \"konnectivity-agent-ncqlz\" (UID: \"eef28aa9-8bf5-4c87-bcb6-245360c85048\") " pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465240 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-node-log\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465256 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgjvg\" (UniqueName: \"kubernetes.io/projected/157ecd5c-7c37-4d11-8ec9-947628a5540d-kube-api-access-fgjvg\") pod \"node-resolver-4qgqw\" (UID: \"157ecd5c-7c37-4d11-8ec9-947628a5540d\") " pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465289 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/be9e984d-1b26-4f1c-8403-aa0790833522-ovn-node-metrics-cert\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465330 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-run-netns\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465317 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/157ecd5c-7c37-4d11-8ec9-947628a5540d-tmp-dir\") pod \"node-resolver-4qgqw\" (UID: \"157ecd5c-7c37-4d11-8ec9-947628a5540d\") " pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465357 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/be9e984d-1b26-4f1c-8403-aa0790833522-ovnkube-script-lib\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465427 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-run-openvswitch\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465475 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-log-socket\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465495 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-run-netns\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465503 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-run-openvswitch\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465500 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-cni-bin\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465541 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-log-socket\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465543 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-cni-bin\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465570 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-slash\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.465775 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465602 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-etc-openvswitch\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465628 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qzh\" (UniqueName: \"kubernetes.io/projected/be9e984d-1b26-4f1c-8403-aa0790833522-kube-api-access-92qzh\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465649 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-slash\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465669 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465696 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/be9e984d-1b26-4f1c-8403-aa0790833522-env-overrides\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eef28aa9-8bf5-4c87-bcb6-245360c85048-konnectivity-ca\") pod \"konnectivity-agent-ncqlz\" (UID: \"eef28aa9-8bf5-4c87-bcb6-245360c85048\") " pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465667 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-etc-openvswitch\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.465802 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be9e984d-1b26-4f1c-8403-aa0790833522-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.466075 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/be9e984d-1b26-4f1c-8403-aa0790833522-ovnkube-script-lib\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.466145 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/be9e984d-1b26-4f1c-8403-aa0790833522-env-overrides\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.466309 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eef28aa9-8bf5-4c87-bcb6-245360c85048-konnectivity-ca\") pod \"konnectivity-agent-ncqlz\" (UID: \"eef28aa9-8bf5-4c87-bcb6-245360c85048\") " pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:07.466956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.466387 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/be9e984d-1b26-4f1c-8403-aa0790833522-ovnkube-config\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.467801 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.467772 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/be9e984d-1b26-4f1c-8403-aa0790833522-ovn-node-metrics-cert\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.468168 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.468146 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eef28aa9-8bf5-4c87-bcb6-245360c85048-agent-certs\") pod \"konnectivity-agent-ncqlz\" (UID: \"eef28aa9-8bf5-4c87-bcb6-245360c85048\") " pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:07.478479 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.478458 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgjvg\" (UniqueName: \"kubernetes.io/projected/157ecd5c-7c37-4d11-8ec9-947628a5540d-kube-api-access-fgjvg\") pod \"node-resolver-4qgqw\" (UID: \"157ecd5c-7c37-4d11-8ec9-947628a5540d\") " pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.478560 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.478520 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92qzh\" (UniqueName: \"kubernetes.io/projected/be9e984d-1b26-4f1c-8403-aa0790833522-kube-api-access-92qzh\") pod \"ovnkube-node-q5j4v\" (UID: \"be9e984d-1b26-4f1c-8403-aa0790833522\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.575968 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.575863 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-55kxj" Aug 04 20:41:07.584968 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.584938 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4004db00_a176_456f_88d5_a8204195d9d3.slice/crio-45f1bd99906d8b9c785bd2780b8129075c018de7bb7945cc9c327ca4a99244bf WatchSource:0}: Error finding container 45f1bd99906d8b9c785bd2780b8129075c018de7bb7945cc9c327ca4a99244bf: Status 404 returned error can't find the container with id 45f1bd99906d8b9c785bd2780b8129075c018de7bb7945cc9c327ca4a99244bf Aug 04 20:41:07.590453 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.590437 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" Aug 04 20:41:07.597867 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.597848 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa9cd51_695a_4ad9_b14b_5fe0dc758268.slice/crio-6c28a480f9b44590c1db5e3c6a650d3846ffa263c57bc0d53a7aa04597c11ec2 WatchSource:0}: Error finding container 6c28a480f9b44590c1db5e3c6a650d3846ffa263c57bc0d53a7aa04597c11ec2: Status 404 returned error can't find the container with id 6c28a480f9b44590c1db5e3c6a650d3846ffa263c57bc0d53a7aa04597c11ec2 Aug 04 20:41:07.610297 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.610278 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" Aug 04 20:41:07.613211 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.613187 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" Aug 04 20:41:07.619240 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.619218 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda20f48cb_c3f8_4a84_8423_fcd9da9ad156.slice/crio-44cb92a1ba90482c6ddb3b09ae2b6663628f8f2c4198d54fc6b59b48c8d90471 WatchSource:0}: Error finding container 44cb92a1ba90482c6ddb3b09ae2b6663628f8f2c4198d54fc6b59b48c8d90471: Status 404 returned error can't find the container with id 44cb92a1ba90482c6ddb3b09ae2b6663628f8f2c4198d54fc6b59b48c8d90471 Aug 04 20:41:07.622938 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.622914 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a71e92_5c9b_470f_83b8_ae4c98ae7a34.slice/crio-d466f36fd023b6b788e44d3f007effd026487b1be3d937e9ae15b3fc2aa1d721 WatchSource:0}: Error finding container d466f36fd023b6b788e44d3f007effd026487b1be3d937e9ae15b3fc2aa1d721: Status 404 returned error can't find the container with id d466f36fd023b6b788e44d3f007effd026487b1be3d937e9ae15b3fc2aa1d721 Aug 04 20:41:07.624329 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.624321 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fdwtb" Aug 04 20:41:07.632292 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.632271 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8bcaf1_8188_447a_84e0_06e13d2cadb2.slice/crio-61ccb342c4aadaecb0a7e1f83035ea7c7a376f5645a7828fc74647ea2ecd1afd WatchSource:0}: Error finding container 61ccb342c4aadaecb0a7e1f83035ea7c7a376f5645a7828fc74647ea2ecd1afd: Status 404 returned error can't find the container with id 61ccb342c4aadaecb0a7e1f83035ea7c7a376f5645a7828fc74647ea2ecd1afd Aug 04 20:41:07.641464 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.641446 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q7c5f" Aug 04 20:41:07.648443 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.648425 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd722b89_197e_48f9_b0a0_33e92b9eea90.slice/crio-b92d4473ebe18b902811597625c51ea7b05970a7531e50f52c61f006b8580f10 WatchSource:0}: Error finding container b92d4473ebe18b902811597625c51ea7b05970a7531e50f52c61f006b8580f10: Status 404 returned error can't find the container with id b92d4473ebe18b902811597625c51ea7b05970a7531e50f52c61f006b8580f10 Aug 04 20:41:07.657873 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.657854 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:07.665004 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.664967 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe9e984d_1b26_4f1c_8403_aa0790833522.slice/crio-1638a132a26ac57ccdc6b541f36623d78f4cf2969e0f93d064260d7a71762d16 WatchSource:0}: Error finding container 1638a132a26ac57ccdc6b541f36623d78f4cf2969e0f93d064260d7a71762d16: Status 404 returned error can't find the container with id 1638a132a26ac57ccdc6b541f36623d78f4cf2969e0f93d064260d7a71762d16 Aug 04 20:41:07.683429 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.683412 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:07.691747 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.691729 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef28aa9_8bf5_4c87_bcb6_245360c85048.slice/crio-0878eab91a068bcb6f5d1ca5cf0a73f11328cbd3e986da68c347d55b7cea00f1 WatchSource:0}: Error finding container 0878eab91a068bcb6f5d1ca5cf0a73f11328cbd3e986da68c347d55b7cea00f1: Status 404 returned error can't find the container with id 0878eab91a068bcb6f5d1ca5cf0a73f11328cbd3e986da68c347d55b7cea00f1 Aug 04 20:41:07.692279 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.692260 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4qgqw" Aug 04 20:41:07.702531 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:07.702505 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157ecd5c_7c37_4d11_8ec9_947628a5540d.slice/crio-6c8a25446d9ecbd2961ede638dddfecc6b42b7e883ce7c6612354e5a8d25e918 WatchSource:0}: Error finding container 6c8a25446d9ecbd2961ede638dddfecc6b42b7e883ce7c6612354e5a8d25e918: Status 404 returned error can't find the container with id 6c8a25446d9ecbd2961ede638dddfecc6b42b7e883ce7c6612354e5a8d25e918 Aug 04 20:41:07.708946 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.708930 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 20:41:07.751062 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.751037 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 20:41:07.868215 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.868132 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:07.868215 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.868168 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:07.868404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:07.868234 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:07.868404 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.868309 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 20:41:07.868404 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.868315 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:07.868404 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.868329 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 20:41:07.868404 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.868339 2576 projected.go:196] Error preparing data for projected volume kube-api-access-d8wn6 for pod openshift-network-diagnostics/network-check-target-lzgsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:07.868404 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.868339 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:07.868404 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.868382 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret podName:a43ca81a-b9a5-45da-b3fa-09452b3296f6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:08.868365816 +0000 UTC m=+3.015236350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret") pod "global-pull-secret-syncer-6j5xp" (UID: "a43ca81a-b9a5-45da-b3fa-09452b3296f6") : object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:07.868404 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.868396 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs podName:e6116bd3-50d9-4d61-95c1-ea0cc53b88ab nodeName:}" failed. No retries permitted until 2026-08-04 20:41:08.868389971 +0000 UTC m=+3.015260506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs") pod "network-metrics-daemon-qhmdn" (UID: "e6116bd3-50d9-4d61-95c1-ea0cc53b88ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:07.868404 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:07.868407 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6 podName:47981c6e-01ce-496b-8d23-7c1c795cdbc6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:08.868401127 +0000 UTC m=+3.015271661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8wn6" (UniqueName: "kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6") pod "network-check-target-lzgsl" (UID: "47981c6e-01ce-496b-8d23-7c1c795cdbc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:08.303487 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.303318 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 20:36:07 +0000 UTC" deadline="2028-04-08 01:54:59.149796983 +0000 UTC" Aug 04 20:41:08.303487 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.303384 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14693h13m50.846417525s" Aug 04 20:41:08.436443 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.436410 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:08.436443 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.436436 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 20:41:08.436692 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:08.436546 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:08.441263 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.441206 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" event={"ID":"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34","Type":"ContainerStarted","Data":"d466f36fd023b6b788e44d3f007effd026487b1be3d937e9ae15b3fc2aa1d721"} Aug 04 20:41:08.446532 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.446501 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" event={"ID":"87987e756c367b8b18190f03114b5471","Type":"ContainerStarted","Data":"c9194213adf630159611a962eb3743397491e454a2a7e8d03de9e8e246d20c43"} Aug 04 20:41:08.449299 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.449270 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" event={"ID":"079dd266070347668559bc450b935f9a","Type":"ContainerStarted","Data":"0adc6343dd48f9dffe64d4e1125d7b52af8207295b7cc3ffb6552a626013be98"} Aug 04 20:41:08.454910 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.454883 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4qgqw" event={"ID":"157ecd5c-7c37-4d11-8ec9-947628a5540d","Type":"ContainerStarted","Data":"6c8a25446d9ecbd2961ede638dddfecc6b42b7e883ce7c6612354e5a8d25e918"} Aug 04 20:41:08.457903 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.457871 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ncqlz" event={"ID":"eef28aa9-8bf5-4c87-bcb6-245360c85048","Type":"ContainerStarted","Data":"0878eab91a068bcb6f5d1ca5cf0a73f11328cbd3e986da68c347d55b7cea00f1"} Aug 04 20:41:08.461153 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.461131 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" event={"ID":"be9e984d-1b26-4f1c-8403-aa0790833522","Type":"ContainerStarted","Data":"1638a132a26ac57ccdc6b541f36623d78f4cf2969e0f93d064260d7a71762d16"} Aug 04 20:41:08.463195 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.463170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q7c5f" event={"ID":"dd722b89-197e-48f9-b0a0-33e92b9eea90","Type":"ContainerStarted","Data":"b92d4473ebe18b902811597625c51ea7b05970a7531e50f52c61f006b8580f10"} Aug 04 20:41:08.468053 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.468026 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fdwtb" event={"ID":"1c8bcaf1-8188-447a-84e0-06e13d2cadb2","Type":"ContainerStarted","Data":"61ccb342c4aadaecb0a7e1f83035ea7c7a376f5645a7828fc74647ea2ecd1afd"} Aug 04 20:41:08.491806 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.491703 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" event={"ID":"a20f48cb-c3f8-4a84-8423-fcd9da9ad156","Type":"ContainerStarted","Data":"44cb92a1ba90482c6ddb3b09ae2b6663628f8f2c4198d54fc6b59b48c8d90471"} Aug 04 20:41:08.500573 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.500543 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" event={"ID":"0aa9cd51-695a-4ad9-b14b-5fe0dc758268","Type":"ContainerStarted","Data":"6c28a480f9b44590c1db5e3c6a650d3846ffa263c57bc0d53a7aa04597c11ec2"} Aug 04 20:41:08.508256 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.508225 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-55kxj" event={"ID":"4004db00-a176-456f-88d5-a8204195d9d3","Type":"ContainerStarted","Data":"45f1bd99906d8b9c785bd2780b8129075c018de7bb7945cc9c327ca4a99244bf"} Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.876080 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.876142 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:08.876194 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:08.876334 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:08.876400 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs podName:e6116bd3-50d9-4d61-95c1-ea0cc53b88ab nodeName:}" failed. No retries permitted until 2026-08-04 20:41:10.876378601 +0000 UTC m=+5.023249139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs") pod "network-metrics-daemon-qhmdn" (UID: "e6116bd3-50d9-4d61-95c1-ea0cc53b88ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:08.876876 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:08.876900 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:08.876913 2576 projected.go:196] Error preparing data for projected volume kube-api-access-d8wn6 for pod openshift-network-diagnostics/network-check-target-lzgsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:08.876965 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6 podName:47981c6e-01ce-496b-8d23-7c1c795cdbc6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:10.876947851 +0000 UTC m=+5.023818389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8wn6" (UniqueName: "kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6") pod "network-check-target-lzgsl" (UID: "47981c6e-01ce-496b-8d23-7c1c795cdbc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:08.877047 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:08.879372 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:08.877085 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret podName:a43ca81a-b9a5-45da-b3fa-09452b3296f6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:10.877073122 +0000 UTC m=+5.023943659 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret") pod "global-pull-secret-syncer-6j5xp" (UID: "a43ca81a-b9a5-45da-b3fa-09452b3296f6") : object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:09.303622 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:09.303529 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 20:36:07 +0000 UTC" deadline="2028-04-09 08:22:06.45499848 +0000 UTC" Aug 04 20:41:09.303622 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:09.303574 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14723h40m57.15142971s" Aug 04 20:41:09.434536 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:09.434499 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:09.434738 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:09.434546 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:09.434738 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:09.434670 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:09.434863 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:09.434791 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:10.436229 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:10.436192 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:10.436728 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:10.436342 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:10.893851 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:10.893769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:10.893851 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:10.893817 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:10.893851 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:10.893852 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:10.894171 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:10.893988 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:10.894171 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:10.894057 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs podName:e6116bd3-50d9-4d61-95c1-ea0cc53b88ab nodeName:}" failed. No retries permitted until 2026-08-04 20:41:14.89403526 +0000 UTC m=+9.040905839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs") pod "network-metrics-daemon-qhmdn" (UID: "e6116bd3-50d9-4d61-95c1-ea0cc53b88ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:10.894171 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:10.894153 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 20:41:10.894363 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:10.894240 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 20:41:10.894363 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:10.894255 2576 projected.go:196] Error preparing data for projected volume kube-api-access-d8wn6 for pod openshift-network-diagnostics/network-check-target-lzgsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:10.894363 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:10.894300 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6 podName:47981c6e-01ce-496b-8d23-7c1c795cdbc6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:14.894285963 +0000 UTC m=+9.041156503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8wn6" (UniqueName: "kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6") pod "network-check-target-lzgsl" (UID: "47981c6e-01ce-496b-8d23-7c1c795cdbc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:10.894517 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:10.894381 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:10.894517 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:10.894421 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret podName:a43ca81a-b9a5-45da-b3fa-09452b3296f6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:14.894408778 +0000 UTC m=+9.041279316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret") pod "global-pull-secret-syncer-6j5xp" (UID: "a43ca81a-b9a5-45da-b3fa-09452b3296f6") : object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:11.434837 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:11.434799 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:11.435043 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:11.434976 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:11.435364 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:11.435341 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:11.435486 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:11.435462 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:12.435149 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:12.435109 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:12.435608 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:12.435255 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:13.433855 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:13.433826 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:13.433855 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:13.433852 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:13.434126 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:13.433968 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:13.434361 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:13.434327 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:14.438138 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:14.437834 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:14.438607 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:14.438256 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:14.935581 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:14.935533 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:14.935785 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:14.935599 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:14.935785 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:14.935635 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:14.935785 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:14.935763 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:14.935989 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:14.935834 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret podName:a43ca81a-b9a5-45da-b3fa-09452b3296f6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:22.935809738 +0000 UTC m=+17.082680275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret") pod "global-pull-secret-syncer-6j5xp" (UID: "a43ca81a-b9a5-45da-b3fa-09452b3296f6") : object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:14.936290 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:14.936169 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:14.936290 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:14.936192 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 20:41:14.936290 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:14.936213 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 20:41:14.936290 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:14.936227 2576 projected.go:196] Error preparing data for projected volume kube-api-access-d8wn6 for pod openshift-network-diagnostics/network-check-target-lzgsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:14.936290 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:14.936243 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs podName:e6116bd3-50d9-4d61-95c1-ea0cc53b88ab nodeName:}" failed. No retries permitted until 2026-08-04 20:41:22.936224852 +0000 UTC m=+17.083095389 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs") pod "network-metrics-daemon-qhmdn" (UID: "e6116bd3-50d9-4d61-95c1-ea0cc53b88ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:14.936290 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:14.936265 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6 podName:47981c6e-01ce-496b-8d23-7c1c795cdbc6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:22.93625364 +0000 UTC m=+17.083124176 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8wn6" (UniqueName: "kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6") pod "network-check-target-lzgsl" (UID: "47981c6e-01ce-496b-8d23-7c1c795cdbc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:15.434419 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:15.434319 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:15.434612 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:15.434483 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:15.434866 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:15.434324 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:15.435039 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:15.434983 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:16.437610 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:16.437553 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:16.438105 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:16.437669 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:17.435411 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:17.434844 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:17.435411 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:17.434866 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:17.435411 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:17.434989 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:17.435411 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:17.435369 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:18.434827 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:18.434745 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:18.435293 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:18.434874 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:19.434620 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:19.434578 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:19.434620 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:19.434615 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:19.434868 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:19.434723 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:19.434868 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:19.434854 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:20.434430 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:20.434342 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:20.434589 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:20.434475 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:21.434852 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:21.434814 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:21.435315 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:21.434864 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:21.435315 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:21.434985 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:21.435315 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:21.435075 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:22.434634 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:22.434605 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:22.434819 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:22.434718 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:23.000202 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:23.000160 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:23.000223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:23.000248 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.000313 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.000373 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.000384 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs podName:e6116bd3-50d9-4d61-95c1-ea0cc53b88ab nodeName:}" failed. No retries permitted until 2026-08-04 20:41:39.000361978 +0000 UTC m=+33.147232548 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs") pod "network-metrics-daemon-qhmdn" (UID: "e6116bd3-50d9-4d61-95c1-ea0cc53b88ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.000400 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.000419 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.000423 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret podName:a43ca81a-b9a5-45da-b3fa-09452b3296f6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:39.000408608 +0000 UTC m=+33.147279173 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret") pod "global-pull-secret-syncer-6j5xp" (UID: "a43ca81a-b9a5-45da-b3fa-09452b3296f6") : object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.000432 2576 projected.go:196] Error preparing data for projected volume kube-api-access-d8wn6 for pod openshift-network-diagnostics/network-check-target-lzgsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:23.000720 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.000469 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6 podName:47981c6e-01ce-496b-8d23-7c1c795cdbc6 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:39.000459622 +0000 UTC m=+33.147330167 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8wn6" (UniqueName: "kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6") pod "network-check-target-lzgsl" (UID: "47981c6e-01ce-496b-8d23-7c1c795cdbc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:23.434164 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:23.434088 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:23.434317 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.434201 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:23.434317 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:23.434262 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:23.434410 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:23.434369 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:24.434120 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:24.434085 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:24.434523 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:24.434218 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:25.434791 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:25.434756 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:25.435242 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:25.434764 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:25.435242 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:25.434922 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:25.435242 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:25.434976 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:26.435215 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:26.435172 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:26.435668 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:26.435285 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:27.433995 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:27.433957 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:27.434185 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:27.433966 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:27.434185 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:27.434119 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:27.434185 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:27.434164 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:28.434592 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:28.434448 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:28.434592 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:28.434547 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:28.568364 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:28.568078 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fdwtb" event={"ID":"1c8bcaf1-8188-447a-84e0-06e13d2cadb2","Type":"ContainerStarted","Data":"133e1ab00ed14e90be10a692f0a082081e0e86e7ef17f1053c3f4dfcf0cfc8b2"} Aug 04 20:41:28.571511 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:28.571455 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" event={"ID":"f0a71e92-5c9b-470f-83b8-ae4c98ae7a34","Type":"ContainerStarted","Data":"7b74c1a544871c1a19841e29dfae3f153c10aa16aa5a892223ecdbf71b4341e7"} Aug 04 20:41:28.572967 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:28.572942 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" event={"ID":"079dd266070347668559bc450b935f9a","Type":"ContainerStarted","Data":"5651e8830cc37e9de38bb12a8089cc12c166b096b2c71b34ed7c3410ee380f75"} Aug 04 20:41:28.574423 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:28.574404 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" event={"ID":"be9e984d-1b26-4f1c-8403-aa0790833522","Type":"ContainerStarted","Data":"f55d8c7023194cc7d098a5753391cb6af083d2478d32a1daa6803718b80eb65f"} Aug 04 20:41:28.587584 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:28.587520 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fdwtb" podStartSLOduration=0.865227904 podStartE2EDuration="21.587501244s" totalImagesPullingTime="20.72227334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:28.587145277 +0000 UTC m=+22.734015832" watchObservedRunningTime="2026-08-04 20:41:28.587501244 +0000 UTC m=+22.734371799" Aug 04 20:41:28.603127 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:28.603088 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bqlbx" podStartSLOduration=0.905948295 podStartE2EDuration="21.603070809s" totalImagesPullingTime="20.697122514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:28.60292194 +0000 UTC m=+22.749792486" watchObservedRunningTime="2026-08-04 20:41:28.603070809 +0000 UTC m=+22.749941366" Aug 04 20:41:29.435085 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.434831 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:29.435808 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.434886 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:29.435808 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:29.435136 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:29.435808 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:29.435176 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:29.578270 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.578235 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" event={"ID":"a20f48cb-c3f8-4a84-8423-fcd9da9ad156","Type":"ContainerStarted","Data":"9def4a2913da09b7aac72847ec301428a1aed56e256dfe27defb540883d13a5b"} Aug 04 20:41:29.579695 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.579668 2576 generic.go:356] "Generic (PLEG): container finished" podID="0aa9cd51-695a-4ad9-b14b-5fe0dc758268" containerID="069eae043a8d83c1d9fd9436eac88f91cdfc3da32968b9338a1b3253573deb82" exitCode=0 Aug 04 20:41:29.579820 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.579746 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" event={"ID":"0aa9cd51-695a-4ad9-b14b-5fe0dc758268","Type":"ContainerDied","Data":"069eae043a8d83c1d9fd9436eac88f91cdfc3da32968b9338a1b3253573deb82"} Aug 04 20:41:29.581224 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.581202 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-55kxj" event={"ID":"4004db00-a176-456f-88d5-a8204195d9d3","Type":"ContainerStarted","Data":"c83db67d246f04e9e69a8b6fc5d4bfd4476c14e6817d731e4cbc5e123ce88c14"} Aug 04 20:41:29.582724 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.582704 2576 generic.go:356] "Generic (PLEG): container finished" podID="87987e756c367b8b18190f03114b5471" containerID="6f1032c73d9af8b144c2803f66e50e6a4db206f8ce424e7ca6fe4137aa1f6ac7" exitCode=0 Aug 04 20:41:29.582790 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.582775 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" event={"ID":"87987e756c367b8b18190f03114b5471","Type":"ContainerDied","Data":"6f1032c73d9af8b144c2803f66e50e6a4db206f8ce424e7ca6fe4137aa1f6ac7"} Aug 04 20:41:29.584465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.584446 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4qgqw" event={"ID":"157ecd5c-7c37-4d11-8ec9-947628a5540d","Type":"ContainerStarted","Data":"2f42e06a87613819532160183b01a84f17962207ac4750db3cef1cc2d660526b"} Aug 04 20:41:29.585846 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.585826 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ncqlz" event={"ID":"eef28aa9-8bf5-4c87-bcb6-245360c85048","Type":"ContainerStarted","Data":"a9afa78911830c12198f826d41aa1fe5b2cb40fd11cbab17f1bb2a063c332e8a"} Aug 04 20:41:29.588627 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.588605 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" event={"ID":"be9e984d-1b26-4f1c-8403-aa0790833522","Type":"ContainerStarted","Data":"51e0a878dc46bcbf186e87e6353f332b89c13aa853a349a4446b60514383be76"} Aug 04 20:41:29.588710 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.588664 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" event={"ID":"be9e984d-1b26-4f1c-8403-aa0790833522","Type":"ContainerStarted","Data":"7bc71701ee8238b78df98c9b7006408215c6ca7559c6016ac4f423c451ebe421"} Aug 04 20:41:29.588710 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.588696 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" event={"ID":"be9e984d-1b26-4f1c-8403-aa0790833522","Type":"ContainerStarted","Data":"d120572f5bc02a417c90fb4a55515b6d72855d1dd689abfc4817c79e238840be"} Aug 04 20:41:29.588816 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.588731 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" event={"ID":"be9e984d-1b26-4f1c-8403-aa0790833522","Type":"ContainerStarted","Data":"03fd2777fd823b59c95baa5deeceec2930fe3c5011576be0c4ef9a7d2cad2bbd"} Aug 04 20:41:29.588816 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.588760 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" event={"ID":"be9e984d-1b26-4f1c-8403-aa0790833522","Type":"ContainerStarted","Data":"630cbfe1e1379c7a95a3f5c654ab0159ad726648adaa54aebf4c92197937c329"} Aug 04 20:41:29.603711 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.603670 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" podStartSLOduration=22.603659788999998 podStartE2EDuration="22.603659789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:28.61704418 +0000 UTC m=+22.763914736" watchObservedRunningTime="2026-08-04 20:41:29.603659789 +0000 UTC m=+23.750530341" Aug 04 20:41:29.621294 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.621257 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4qgqw" podStartSLOduration=2.02030872 podStartE2EDuration="22.621246161s" totalImagesPullingTime="20.600937441s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:29.621034937 +0000 UTC m=+23.767905502" watchObservedRunningTime="2026-08-04 20:41:29.621246161 +0000 UTC m=+23.768116716" Aug 04 20:41:29.657965 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.657861 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ncqlz" podStartSLOduration=2.04646131 podStartE2EDuration="22.657847811s" totalImagesPullingTime="20.611386501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:29.657195075 +0000 UTC m=+23.804065630" watchObservedRunningTime="2026-08-04 20:41:29.657847811 +0000 UTC m=+23.804718365" Aug 04 20:41:29.674812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.674776 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-55kxj" podStartSLOduration=1.954773884 podStartE2EDuration="22.6747626s" totalImagesPullingTime="20.719988716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:29.673887399 +0000 UTC m=+23.820757954" watchObservedRunningTime="2026-08-04 20:41:29.6747626 +0000 UTC m=+23.821633155" Aug 04 20:41:29.892456 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.892417 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:29.893018 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.893000 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:29.936680 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:29.936649 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:30.005619 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.005532 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:30.434380 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.434313 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:30.434536 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:30.434403 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:30.593824 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.593784 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" event={"ID":"87987e756c367b8b18190f03114b5471","Type":"ContainerStarted","Data":"2521c2441536f23a149617c764655bcbde7f7673f4897ee8758870dc9b321fd0"} Aug 04 20:41:30.601114 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.599384 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q7c5f" event={"ID":"dd722b89-197e-48f9-b0a0-33e92b9eea90","Type":"ContainerStarted","Data":"efbe95925a7925dd1fda41333d71389765edd1c88668f5faa072df35b5d62174"} Aug 04 20:41:30.601114 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.599438 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:30.601114 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.601063 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ncqlz" Aug 04 20:41:30.611065 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.611014 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" podStartSLOduration=23.610995514 podStartE2EDuration="23.610995514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:30.610161732 +0000 UTC m=+24.757032287" watchObservedRunningTime="2026-08-04 20:41:30.610995514 +0000 UTC m=+24.757866069" Aug 04 20:41:30.625202 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.625160 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q7c5f" podStartSLOduration=2.961549169 podStartE2EDuration="23.62514628s" totalImagesPullingTime="20.663597111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:30.624920101 +0000 UTC m=+24.771790647" watchObservedRunningTime="2026-08-04 20:41:30.62514628 +0000 UTC m=+24.772016836" Aug 04 20:41:30.662863 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.662841 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 20:41:30.747487 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.747404 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:30.847573 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:30.847517 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:31.334850 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.334705 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T20:41:30.662859969Z","UUID":"6426d331-e6c3-4616-9a02-84d415fc5a62","Handler":null,"Name":"","Endpoint":""} Aug 04 20:41:31.336318 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.336296 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 20:41:31.336451 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.336341 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 20:41:31.434135 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.434101 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:31.434304 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.434145 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:31.434304 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:31.434230 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:31.434420 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:31.434316 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:31.581474 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.581434 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:31.605375 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.605305 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" event={"ID":"be9e984d-1b26-4f1c-8403-aa0790833522","Type":"ContainerStarted","Data":"f79839b5832cabcc517ea92ed896cbdf213b1e85095dbe35fbee122151c3c45d"} Aug 04 20:41:31.607485 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.607458 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" event={"ID":"a20f48cb-c3f8-4a84-8423-fcd9da9ad156","Type":"ContainerStarted","Data":"b548a67558ea7ee83fa19d2a78d3c56ae055dad790251de8a3421bc1b762bfb5"} Aug 04 20:41:31.656100 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.656064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:31.728193 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:31.728157 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:32.123777 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:32.123752 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:32.227010 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:32.226974 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:32.434427 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:32.434348 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:32.434583 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:32.434463 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:32.612023 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:32.611987 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" event={"ID":"a20f48cb-c3f8-4a84-8423-fcd9da9ad156","Type":"ContainerStarted","Data":"fe7d104e05a2eb9034f3db35d2ce2f58c880b32efcaf5c4f4b5a221177951bd1"} Aug 04 20:41:33.434038 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:33.433998 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:33.434038 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:33.434037 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:33.434278 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:33.434136 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:33.434342 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:33.434274 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:34.434419 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:34.434268 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:34.434758 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:34.434509 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:34.619639 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:34.619610 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" event={"ID":"be9e984d-1b26-4f1c-8403-aa0790833522","Type":"ContainerStarted","Data":"6deb3e7b9e66e6b69f7c3cfd94cb3c1c21b581ebc91857424e8d1872dd056566"} Aug 04 20:41:34.623251 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:34.623166 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:34.623251 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:34.623248 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:34.644038 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:34.644009 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:34.668633 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:34.668578 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" podStartSLOduration=6.962314657 podStartE2EDuration="27.668560234s" totalImagesPullingTime="20.706245577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:34.668193821 +0000 UTC m=+28.815064376" watchObservedRunningTime="2026-08-04 20:41:34.668560234 +0000 UTC m=+28.815430790" Aug 04 20:41:34.669256 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:34.669216 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpczc" podStartSLOduration=4.349741885 podStartE2EDuration="27.6691994s" totalImagesPullingTime="23.319457515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:32.635464297 +0000 UTC m=+26.782334851" watchObservedRunningTime="2026-08-04 20:41:34.6691994 +0000 UTC m=+28.816069962" Aug 04 20:41:35.434803 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:35.434774 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:35.434803 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:35.434813 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:35.435495 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:35.434897 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:35.435495 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:35.435012 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:35.623548 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:35.623513 2576 generic.go:356] "Generic (PLEG): container finished" podID="0aa9cd51-695a-4ad9-b14b-5fe0dc758268" containerID="3ef66ceef2a4fea0051de5ea905628fb515e993c846db64c7df6f6e211e5f8d7" exitCode=0 Aug 04 20:41:35.623777 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:35.623590 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" event={"ID":"0aa9cd51-695a-4ad9-b14b-5fe0dc758268","Type":"ContainerDied","Data":"3ef66ceef2a4fea0051de5ea905628fb515e993c846db64c7df6f6e211e5f8d7"} Aug 04 20:41:35.623777 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:35.623778 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:35.639787 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:35.639769 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:41:36.435631 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:36.435443 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:36.436059 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:36.435684 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:36.714513 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:36.714422 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6j5xp"] Aug 04 20:41:36.714639 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:36.714517 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:36.714678 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:36.714631 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:36.724797 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:36.724772 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lzgsl"] Aug 04 20:41:36.724972 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:36.724856 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:36.725028 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:36.724963 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:36.726142 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:36.726117 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qhmdn"] Aug 04 20:41:36.726318 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:36.726210 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:36.726318 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:36.726294 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:37.629738 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:37.629702 2576 generic.go:356] "Generic (PLEG): container finished" podID="0aa9cd51-695a-4ad9-b14b-5fe0dc758268" containerID="d90e965ebb82882c71ade8554bca35927620b68d06ffec40cfa2bc53e189ceeb" exitCode=0 Aug 04 20:41:37.630198 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:37.629790 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" event={"ID":"0aa9cd51-695a-4ad9-b14b-5fe0dc758268","Type":"ContainerDied","Data":"d90e965ebb82882c71ade8554bca35927620b68d06ffec40cfa2bc53e189ceeb"} Aug 04 20:41:38.433836 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:38.433801 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:38.434016 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:38.433816 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:38.434016 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:38.433931 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:38.434016 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:38.434010 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:38.434136 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:38.434006 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:38.434136 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:38.434123 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:39.023397 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:39.023310 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:39.023397 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:39.023349 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:39.023397 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:39.023379 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:39.023800 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:39.023476 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 20:41:39.023800 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:39.023484 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:39.023800 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:39.023500 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 20:41:39.023800 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:39.023511 2576 projected.go:196] Error preparing data for projected volume kube-api-access-d8wn6 for pod openshift-network-diagnostics/network-check-target-lzgsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:39.023800 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:39.023515 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:39.023800 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:39.023550 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs podName:e6116bd3-50d9-4d61-95c1-ea0cc53b88ab nodeName:}" failed. No retries permitted until 2026-08-04 20:42:11.023532946 +0000 UTC m=+65.170403501 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs") pod "network-metrics-daemon-qhmdn" (UID: "e6116bd3-50d9-4d61-95c1-ea0cc53b88ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 20:41:39.023800 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:39.023565 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6 podName:47981c6e-01ce-496b-8d23-7c1c795cdbc6 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:11.023558823 +0000 UTC m=+65.170429357 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8wn6" (UniqueName: "kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6") pod "network-check-target-lzgsl" (UID: "47981c6e-01ce-496b-8d23-7c1c795cdbc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 20:41:39.023800 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:39.023576 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret podName:a43ca81a-b9a5-45da-b3fa-09452b3296f6 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:11.023571356 +0000 UTC m=+65.170441888 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret") pod "global-pull-secret-syncer-6j5xp" (UID: "a43ca81a-b9a5-45da-b3fa-09452b3296f6") : object "kube-system"/"original-pull-secret" not registered Aug 04 20:41:39.637841 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:39.637808 2576 generic.go:356] "Generic (PLEG): container finished" podID="0aa9cd51-695a-4ad9-b14b-5fe0dc758268" containerID="88bb490ad9ebe3fca4e53f414106263638b274e2f427e7e6bfb2867ec5b0d168" exitCode=0 Aug 04 20:41:39.638014 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:39.637877 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" event={"ID":"0aa9cd51-695a-4ad9-b14b-5fe0dc758268","Type":"ContainerDied","Data":"88bb490ad9ebe3fca4e53f414106263638b274e2f427e7e6bfb2867ec5b0d168"} Aug 04 20:41:40.434560 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:40.434524 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:40.434975 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:40.434525 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:40.434975 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:40.434632 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6j5xp" podUID="a43ca81a-b9a5-45da-b3fa-09452b3296f6" Aug 04 20:41:40.434975 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:40.434525 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:40.434975 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:40.434701 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzgsl" podUID="47981c6e-01ce-496b-8d23-7c1c795cdbc6" Aug 04 20:41:40.434975 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:40.434773 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmdn" podUID="e6116bd3-50d9-4d61-95c1-ea0cc53b88ab" Aug 04 20:41:41.185342 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.185312 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeReady" Aug 04 20:41:41.185546 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.185459 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 20:41:41.245197 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.245163 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mn9mw"] Aug 04 20:41:41.247485 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.247449 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.247752 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.247732 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7cc5c"] Aug 04 20:41:41.249598 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.249573 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7cc5c" Aug 04 20:41:41.250314 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.250112 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 20:41:41.250643 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.250615 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 20:41:41.250810 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.250790 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kcgl6\"" Aug 04 20:41:41.253631 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.251949 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 20:41:41.253631 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.252144 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 20:41:41.253631 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.252636 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q6crk\"" Aug 04 20:41:41.253631 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.253078 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 20:41:41.263985 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.263960 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mn9mw"] Aug 04 20:41:41.264533 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.264514 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7cc5c"] Aug 04 20:41:41.340418 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.340380 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjxxk\" (UniqueName: \"kubernetes.io/projected/cd2bb155-816d-4b33-9c0f-528c36d56f55-kube-api-access-qjxxk\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.340569 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.340439 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qvqn\" (UniqueName: \"kubernetes.io/projected/699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6-kube-api-access-9qvqn\") pod \"ingress-canary-7cc5c\" (UID: \"699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6\") " pod="openshift-ingress-canary/ingress-canary-7cc5c" Aug 04 20:41:41.340569 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.340467 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cd2bb155-816d-4b33-9c0f-528c36d56f55-config-volume\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.340569 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.340506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.340569 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.340529 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2bb155-816d-4b33-9c0f-528c36d56f55-tmp-dir\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.340569 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.340548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6-cert\") pod \"ingress-canary-7cc5c\" (UID: \"699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6\") " pod="openshift-ingress-canary/ingress-canary-7cc5c" Aug 04 20:41:41.396188 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.396158 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:41.441504 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.441417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjxxk\" (UniqueName: \"kubernetes.io/projected/cd2bb155-816d-4b33-9c0f-528c36d56f55-kube-api-access-qjxxk\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.441504 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.441491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qvqn\" (UniqueName: \"kubernetes.io/projected/699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6-kube-api-access-9qvqn\") pod \"ingress-canary-7cc5c\" (UID: \"699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6\") " pod="openshift-ingress-canary/ingress-canary-7cc5c" Aug 04 20:41:41.442004 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.441529 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cd2bb155-816d-4b33-9c0f-528c36d56f55-config-volume\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.442004 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.441676 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.442004 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.441707 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2bb155-816d-4b33-9c0f-528c36d56f55-tmp-dir\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.442004 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.441734 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6-cert\") pod \"ingress-canary-7cc5c\" (UID: \"699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6\") " pod="openshift-ingress-canary/ingress-canary-7cc5c" Aug 04 20:41:41.442004 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:41.441768 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 20:41:41.442004 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:41.441834 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls podName:cd2bb155-816d-4b33-9c0f-528c36d56f55 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:41.941814368 +0000 UTC m=+36.088684904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls") pod "dns-default-mn9mw" (UID: "cd2bb155-816d-4b33-9c0f-528c36d56f55") : secret "dns-default-metrics-tls" not found Aug 04 20:41:41.442211 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.442045 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2bb155-816d-4b33-9c0f-528c36d56f55-tmp-dir\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.442211 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.442155 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cd2bb155-816d-4b33-9c0f-528c36d56f55-config-volume\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.447150 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.446999 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6-cert\") pod \"ingress-canary-7cc5c\" (UID: \"699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6\") " pod="openshift-ingress-canary/ingress-canary-7cc5c" Aug 04 20:41:41.453361 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.453340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjxxk\" (UniqueName: \"kubernetes.io/projected/cd2bb155-816d-4b33-9c0f-528c36d56f55-kube-api-access-qjxxk\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.453478 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.453389 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qvqn\" (UniqueName: \"kubernetes.io/projected/699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6-kube-api-access-9qvqn\") pod \"ingress-canary-7cc5c\" (UID: \"699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6\") " pod="openshift-ingress-canary/ingress-canary-7cc5c" Aug 04 20:41:41.495713 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.495675 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:41.571224 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.571185 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7cc5c" Aug 04 20:41:41.725111 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.725032 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7cc5c"] Aug 04 20:41:41.730264 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:41.730240 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod699b5cd2_1fb4_4fb3_a6c0_1bf61957f8c6.slice/crio-ace1b55fecd162bf8cffee7676689acfb3480c7891bb0547344be08d9954ef75 WatchSource:0}: Error finding container ace1b55fecd162bf8cffee7676689acfb3480c7891bb0547344be08d9954ef75: Status 404 returned error can't find the container with id ace1b55fecd162bf8cffee7676689acfb3480c7891bb0547344be08d9954ef75 Aug 04 20:41:41.946508 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:41.946469 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:41.946669 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:41.946605 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 20:41:41.946669 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:41.946663 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls podName:cd2bb155-816d-4b33-9c0f-528c36d56f55 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:42.946647777 +0000 UTC m=+37.093518310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls") pod "dns-default-mn9mw" (UID: "cd2bb155-816d-4b33-9c0f-528c36d56f55") : secret "dns-default-metrics-tls" not found Aug 04 20:41:42.436140 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.436108 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:41:42.436529 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.436508 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:41:42.436941 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.436922 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:41:42.439533 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.439512 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 20:41:42.439812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.439790 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 20:41:42.440740 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.440724 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 20:41:42.440740 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.440737 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nn596\"" Aug 04 20:41:42.441015 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.441000 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 20:41:42.441206 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.441191 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8x4x\"" Aug 04 20:41:42.650036 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.650000 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7cc5c" event={"ID":"699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6","Type":"ContainerStarted","Data":"ace1b55fecd162bf8cffee7676689acfb3480c7891bb0547344be08d9954ef75"} Aug 04 20:41:42.956115 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:42.956079 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:42.956303 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:42.956219 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 20:41:42.956303 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:42.956287 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls podName:cd2bb155-816d-4b33-9c0f-528c36d56f55 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:44.956268248 +0000 UTC m=+39.103138784 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls") pod "dns-default-mn9mw" (UID: "cd2bb155-816d-4b33-9c0f-528c36d56f55") : secret "dns-default-metrics-tls" not found Aug 04 20:41:44.974426 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:44.974387 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:44.975040 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:44.974557 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 20:41:44.975040 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:44.974628 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls podName:cd2bb155-816d-4b33-9c0f-528c36d56f55 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:48.974612873 +0000 UTC m=+43.121483406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls") pod "dns-default-mn9mw" (UID: "cd2bb155-816d-4b33-9c0f-528c36d56f55") : secret "dns-default-metrics-tls" not found Aug 04 20:41:45.660028 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:45.659813 2576 generic.go:356] "Generic (PLEG): container finished" podID="0aa9cd51-695a-4ad9-b14b-5fe0dc758268" containerID="170631b9bbae7bf0ec451759894b85f4dd7cd1e7297536607f2d94831237e3e8" exitCode=0 Aug 04 20:41:45.660164 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:45.659851 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" event={"ID":"0aa9cd51-695a-4ad9-b14b-5fe0dc758268","Type":"ContainerDied","Data":"170631b9bbae7bf0ec451759894b85f4dd7cd1e7297536607f2d94831237e3e8"} Aug 04 20:41:45.661549 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:45.661526 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7cc5c" event={"ID":"699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6","Type":"ContainerStarted","Data":"fe8d37a44c4299619820ff51f9c3e1e5d1e8cd8cc13d17b53aaf6c7a19c43818"} Aug 04 20:41:45.699524 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:45.699481 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7cc5c" podStartSLOduration=1.195830251 podStartE2EDuration="4.699465482s" totalImagesPullingTime="3.503635231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:45.698874203 +0000 UTC m=+39.845744762" watchObservedRunningTime="2026-08-04 20:41:45.699465482 +0000 UTC m=+39.846336036" Aug 04 20:41:46.666996 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:46.666964 2576 generic.go:356] "Generic (PLEG): container finished" podID="0aa9cd51-695a-4ad9-b14b-5fe0dc758268" containerID="a21ba8314a7891b60a400293582f84426822b62751258b6eb9d786641cb09e15" exitCode=0 Aug 04 20:41:46.667350 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:46.667041 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" event={"ID":"0aa9cd51-695a-4ad9-b14b-5fe0dc758268","Type":"ContainerDied","Data":"a21ba8314a7891b60a400293582f84426822b62751258b6eb9d786641cb09e15"} Aug 04 20:41:46.672700 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:46.672674 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:46.772573 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:46.772546 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:47.672703 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:47.672668 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" event={"ID":"0aa9cd51-695a-4ad9-b14b-5fe0dc758268","Type":"ContainerStarted","Data":"d0ce3b95059bdecf49c53b5b2bb1b69671827ff9260947fd5a2611f5afe914cc"} Aug 04 20:41:47.698838 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:47.698786 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cbt9z" podStartSLOduration=7.035230065 podStartE2EDuration="40.698771662s" totalImagesPullingTime="33.663541597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:47.697187013 +0000 UTC m=+41.844057567" watchObservedRunningTime="2026-08-04 20:41:47.698771662 +0000 UTC m=+41.845642216" Aug 04 20:41:47.830778 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:47.830747 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:47.894978 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:47.894954 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:49.001430 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:49.001382 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:49.001806 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:49.001523 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 20:41:49.001806 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:49.001586 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls podName:cd2bb155-816d-4b33-9c0f-528c36d56f55 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:57.0015713 +0000 UTC m=+51.148441833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls") pod "dns-default-mn9mw" (UID: "cd2bb155-816d-4b33-9c0f-528c36d56f55") : secret "dns-default-metrics-tls" not found Aug 04 20:41:53.557158 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:53.557100 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:53.618329 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:53.618279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:57.048050 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:57.048017 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:41:57.048433 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:57.048170 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 20:41:57.048433 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:41:57.048236 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls podName:cd2bb155-816d-4b33-9c0f-528c36d56f55 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:13.04821956 +0000 UTC m=+67.195090092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls") pod "dns-default-mn9mw" (UID: "cd2bb155-816d-4b33-9c0f-528c36d56f55") : secret "dns-default-metrics-tls" not found Aug 04 20:41:58.240520 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.240483 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-6slt9"] Aug 04 20:41:58.264358 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.264327 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.264677 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.264573 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-6slt9"] Aug 04 20:41:58.269622 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.269573 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Aug 04 20:41:58.269622 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.269587 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Aug 04 20:41:58.269622 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.269602 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7bc8m\"" Aug 04 20:41:58.269824 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.269641 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 20:41:58.270125 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.270093 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 20:41:58.276255 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.276235 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Aug 04 20:41:58.355378 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.355348 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-snapshots\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.355612 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.355387 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-trusted-ca-bundle\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.355612 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.355405 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-serving-cert\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.355612 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.355421 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-tmp\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.355612 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.355469 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-service-ca-bundle\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.355612 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.355549 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxz8m\" (UniqueName: \"kubernetes.io/projected/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-kube-api-access-lxz8m\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.456815 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.456788 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-snapshots\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.456941 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.456827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-trusted-ca-bundle\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.456941 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.456845 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-serving-cert\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.456941 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.456872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-tmp\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.456941 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.456894 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-service-ca-bundle\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.457098 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.456972 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxz8m\" (UniqueName: \"kubernetes.io/projected/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-kube-api-access-lxz8m\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.457421 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.457399 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-tmp\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.457522 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.457503 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-snapshots\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.457626 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.457593 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-service-ca-bundle\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.457828 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.457809 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-trusted-ca-bundle\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.460454 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.460435 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-serving-cert\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.472108 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.472082 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxz8m\" (UniqueName: \"kubernetes.io/projected/8bf442c4-3348-46ae-ac9c-69b05f9dcbfb-kube-api-access-lxz8m\") pod \"insights-operator-77696685b-6slt9\" (UID: \"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb\") " pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.549576 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.549524 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:58.577741 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.577720 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-6slt9" Aug 04 20:41:58.620401 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.620372 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:41:58.714211 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:58.714181 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-6slt9"] Aug 04 20:41:58.716802 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:41:58.716781 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf442c4_3348_46ae_ac9c_69b05f9dcbfb.slice/crio-79e1aec6ef98cd231464c1c67c563903eca0a260614f562ac53f8b254e576767 WatchSource:0}: Error finding container 79e1aec6ef98cd231464c1c67c563903eca0a260614f562ac53f8b254e576767: Status 404 returned error can't find the container with id 79e1aec6ef98cd231464c1c67c563903eca0a260614f562ac53f8b254e576767 Aug 04 20:41:59.701244 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:41:59.701197 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-6slt9" event={"ID":"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb","Type":"ContainerStarted","Data":"79e1aec6ef98cd231464c1c67c563903eca0a260614f562ac53f8b254e576767"} Aug 04 20:42:01.708075 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.708038 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-6slt9" event={"ID":"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb","Type":"ContainerStarted","Data":"13bde20f410650d7cd15d30809c446f61597cb3659c687a0da624fbc60079825"} Aug 04 20:42:01.732072 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.732018 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-6slt9" podStartSLOduration=1.629005554 podStartE2EDuration="3.731999827s" totalImagesPullingTime="2.102994273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:01.730850709 +0000 UTC m=+55.877721263" watchObservedRunningTime="2026-08-04 20:42:01.731999827 +0000 UTC m=+55.878870381" Aug 04 20:42:01.905794 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.905739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:01.916245 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.916221 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-27h2f"] Aug 04 20:42:01.920244 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.920218 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:01.923239 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.923219 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d7shv\"" Aug 04 20:42:01.925000 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.924978 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 20:42:01.925703 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.925685 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 20:42:01.939837 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.939811 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-27h2f"] Aug 04 20:42:01.978923 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.978833 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:01.985867 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.985847 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:01.985937 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.985890 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:01.985983 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.985969 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-crio-socket\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:01.986018 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.986007 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wbm9\" (UniqueName: \"kubernetes.io/projected/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-api-access-2wbm9\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:01.986050 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:01.986035 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-data-volume\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.086900 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.086862 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.087064 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.086923 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-crio-socket\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.087064 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.086968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wbm9\" (UniqueName: \"kubernetes.io/projected/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-api-access-2wbm9\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.087064 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.086991 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-data-volume\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.087064 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:02.087001 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 20:42:02.087064 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.087035 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.087261 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:02.087071 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:02.587055758 +0000 UTC m=+56.733926291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : secret "insights-runtime-extractor-tls" not found Aug 04 20:42:02.087261 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:02.087103 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 20:42:02.087261 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.087008 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-crio-socket\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.087261 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:02.087163 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:02.587149387 +0000 UTC m=+56.734019937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : configmap "kube-rbac-proxy" not found Aug 04 20:42:02.087397 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.087304 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-data-volume\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.096812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.096783 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wbm9\" (UniqueName: \"kubernetes.io/projected/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-api-access-2wbm9\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.592493 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.592446 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.592671 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:02.592539 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:02.592671 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:02.592578 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 20:42:02.592671 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:02.592651 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:03.592634711 +0000 UTC m=+57.739505246 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : configmap "kube-rbac-proxy" not found Aug 04 20:42:02.592671 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:02.592663 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 20:42:02.592808 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:02.592702 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:03.592690893 +0000 UTC m=+57.739561425 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : secret "insights-runtime-extractor-tls" not found Aug 04 20:42:03.214202 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.214172 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx"] Aug 04 20:42:03.216241 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.216225 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx" Aug 04 20:42:03.218620 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.218595 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kd2zz\"" Aug 04 20:42:03.218736 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.218595 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Aug 04 20:42:03.219504 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.219490 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Aug 04 20:42:03.227085 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.227067 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx"] Aug 04 20:42:03.298415 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.298375 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5l5m\" (UniqueName: \"kubernetes.io/projected/76417d59-9d65-441d-92d7-c5f49a592c60-kube-api-access-q5l5m\") pod \"volume-data-source-validator-5b8776669-9jtjx\" (UID: \"76417d59-9d65-441d-92d7-c5f49a592c60\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx" Aug 04 20:42:03.318340 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.318314 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z"] Aug 04 20:42:03.320380 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.320365 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:03.323615 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.323594 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Aug 04 20:42:03.323723 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.323599 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Aug 04 20:42:03.323723 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.323713 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Aug 04 20:42:03.323813 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.323804 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gsf5n\"" Aug 04 20:42:03.328869 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.328852 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t"] Aug 04 20:42:03.330791 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.330777 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.334531 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.334479 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 04 20:42:03.334666 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.334648 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Aug 04 20:42:03.334723 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.334695 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Aug 04 20:42:03.334763 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.334747 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Aug 04 20:42:03.334800 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.334648 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Aug 04 20:42:03.334975 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.334962 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 04 20:42:03.335031 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.334962 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 04 20:42:03.343011 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.342988 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z"] Aug 04 20:42:03.347144 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.347124 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t"] Aug 04 20:42:03.398708 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.398684 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-hub\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.398863 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.398711 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.398863 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.398740 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5l5m\" (UniqueName: \"kubernetes.io/projected/76417d59-9d65-441d-92d7-c5f49a592c60-kube-api-access-q5l5m\") pod \"volume-data-source-validator-5b8776669-9jtjx\" (UID: \"76417d59-9d65-441d-92d7-c5f49a592c60\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx" Aug 04 20:42:03.398863 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.398802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl2vh\" (UniqueName: \"kubernetes.io/projected/f7ea2c4d-1554-402c-b432-002d85830ce9-kube-api-access-dl2vh\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:03.399068 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.398931 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c033e8-fff1-425a-9166-632197c56a76-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.399068 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.398968 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt7g8\" (UniqueName: \"kubernetes.io/projected/d3c033e8-fff1-425a-9166-632197c56a76-kube-api-access-mt7g8\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.399068 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.399010 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-ca\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.399068 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.399038 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:03.399068 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.399064 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.408793 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.408774 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5l5m\" (UniqueName: \"kubernetes.io/projected/76417d59-9d65-441d-92d7-c5f49a592c60-kube-api-access-q5l5m\") pod \"volume-data-source-validator-5b8776669-9jtjx\" (UID: \"76417d59-9d65-441d-92d7-c5f49a592c60\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx" Aug 04 20:42:03.503535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.500692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-hub\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.503535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.500739 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.503535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.500786 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl2vh\" (UniqueName: \"kubernetes.io/projected/f7ea2c4d-1554-402c-b432-002d85830ce9-kube-api-access-dl2vh\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:03.503535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.500934 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c033e8-fff1-425a-9166-632197c56a76-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.503535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.500968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt7g8\" (UniqueName: \"kubernetes.io/projected/d3c033e8-fff1-425a-9166-632197c56a76-kube-api-access-mt7g8\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.503535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.501010 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-ca\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.503535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.501045 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:03.503535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.501075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.504419 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:03.504060 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 20:42:03.504419 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:03.504155 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls podName:f7ea2c4d-1554-402c-b432-002d85830ce9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:04.00412914 +0000 UTC m=+58.150999691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-nrj2z" (UID: "f7ea2c4d-1554-402c-b432-002d85830ce9") : secret "samples-operator-tls" not found Aug 04 20:42:03.504419 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.504160 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.504618 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.504451 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c033e8-fff1-425a-9166-632197c56a76-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.506233 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.506209 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-hub\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.506548 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.506529 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.506591 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.506530 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d3c033e8-fff1-425a-9166-632197c56a76-ca\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.513374 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.513342 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt7g8\" (UniqueName: \"kubernetes.io/projected/d3c033e8-fff1-425a-9166-632197c56a76-kube-api-access-mt7g8\") pod \"cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t\" (UID: \"d3c033e8-fff1-425a-9166-632197c56a76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.513789 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.513760 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl2vh\" (UniqueName: \"kubernetes.io/projected/f7ea2c4d-1554-402c-b432-002d85830ce9-kube-api-access-dl2vh\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:03.525453 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.525433 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx" Aug 04 20:42:03.603110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.601872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:03.603110 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.601960 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:03.603110 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:03.602111 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 20:42:03.603110 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:03.602175 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:05.60215524 +0000 UTC m=+59.749025776 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : secret "insights-runtime-extractor-tls" not found Aug 04 20:42:03.603110 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:03.602587 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 20:42:03.603110 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:03.602642 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:05.602624233 +0000 UTC m=+59.749494769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : configmap "kube-rbac-proxy" not found Aug 04 20:42:03.641983 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.641962 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4qgqw_157ecd5c-7c37-4d11-8ec9-947628a5540d/dns-node-resolver/0.log" Aug 04 20:42:03.655136 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.655112 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:42:03.660331 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.660310 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx"] Aug 04 20:42:03.663515 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:03.663494 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76417d59_9d65_441d_92d7_c5f49a592c60.slice/crio-661b9fb3a7333dcf7ea5b65e68a4512a3ecad33650353d16d6bd28c6d790bba7 WatchSource:0}: Error finding container 661b9fb3a7333dcf7ea5b65e68a4512a3ecad33650353d16d6bd28c6d790bba7: Status 404 returned error can't find the container with id 661b9fb3a7333dcf7ea5b65e68a4512a3ecad33650353d16d6bd28c6d790bba7 Aug 04 20:42:03.714570 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.714535 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx" event={"ID":"76417d59-9d65-441d-92d7-c5f49a592c60","Type":"ContainerStarted","Data":"661b9fb3a7333dcf7ea5b65e68a4512a3ecad33650353d16d6bd28c6d790bba7"} Aug 04 20:42:03.790337 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:03.790274 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t"] Aug 04 20:42:03.793363 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:03.793337 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3c033e8_fff1_425a_9166_632197c56a76.slice/crio-66f3c9719b470160743de86a80e3f03054166cd8edf86b6baaf0b00b5aa25482 WatchSource:0}: Error finding container 66f3c9719b470160743de86a80e3f03054166cd8edf86b6baaf0b00b5aa25482: Status 404 returned error can't find the container with id 66f3c9719b470160743de86a80e3f03054166cd8edf86b6baaf0b00b5aa25482 Aug 04 20:42:04.005250 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:04.005213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:04.005414 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:04.005326 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 20:42:04.005414 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:04.005382 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls podName:f7ea2c4d-1554-402c-b432-002d85830ce9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:05.005368218 +0000 UTC m=+59.152238750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-nrj2z" (UID: "f7ea2c4d-1554-402c-b432-002d85830ce9") : secret "samples-operator-tls" not found Aug 04 20:42:04.446463 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:04.446434 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-55kxj_4004db00-a176-456f-88d5-a8204195d9d3/node-ca/0.log" Aug 04 20:42:04.719271 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:04.719176 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" event={"ID":"d3c033e8-fff1-425a-9166-632197c56a76","Type":"ContainerStarted","Data":"66f3c9719b470160743de86a80e3f03054166cd8edf86b6baaf0b00b5aa25482"} Aug 04 20:42:05.014593 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:05.014511 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:05.014750 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:05.014635 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 20:42:05.014750 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:05.014685 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls podName:f7ea2c4d-1554-402c-b432-002d85830ce9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:07.014672026 +0000 UTC m=+61.161542559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-nrj2z" (UID: "f7ea2c4d-1554-402c-b432-002d85830ce9") : secret "samples-operator-tls" not found Aug 04 20:42:05.442772 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:05.442730 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7cc5c_699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6/serve-healthcheck-canary/0.log" Aug 04 20:42:05.621237 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:05.621203 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:05.621667 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:05.621260 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:05.621667 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:05.621361 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 20:42:05.621667 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:05.621386 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 20:42:05.621667 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:05.621438 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:09.621417951 +0000 UTC m=+63.768288512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : configmap "kube-rbac-proxy" not found Aug 04 20:42:05.621667 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:05.621462 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:09.621452288 +0000 UTC m=+63.768322822 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : secret "insights-runtime-extractor-tls" not found Aug 04 20:42:06.727141 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:06.727097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx" event={"ID":"76417d59-9d65-441d-92d7-c5f49a592c60","Type":"ContainerStarted","Data":"5d2f9fb075637463fa4a20c8fb7cb94fc164a7e736d1d09790d101ac0f2b2489"} Aug 04 20:42:06.747061 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:06.747009 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9jtjx" podStartSLOduration=1.333036319 podStartE2EDuration="3.746992021s" totalImagesPullingTime="2.413955702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:06.745182954 +0000 UTC m=+60.892053510" watchObservedRunningTime="2026-08-04 20:42:06.746992021 +0000 UTC m=+60.893862575" Aug 04 20:42:06.838340 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:06.838292 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:06.906232 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:06.906203 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:07.032614 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:07.032532 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:07.032775 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:07.032697 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 20:42:07.032846 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:07.032778 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls podName:f7ea2c4d-1554-402c-b432-002d85830ce9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:11.032756735 +0000 UTC m=+65.179627283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-nrj2z" (UID: "f7ea2c4d-1554-402c-b432-002d85830ce9") : secret "samples-operator-tls" not found Aug 04 20:42:07.647246 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:07.647214 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5j4v" Aug 04 20:42:07.781582 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:07.781546 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:07.880619 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:07.880590 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:08.737851 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:08.737812 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" event={"ID":"d3c033e8-fff1-425a-9166-632197c56a76","Type":"ContainerStarted","Data":"6abda1d2363e7a22b21c8fa8403a106dc2be25b01c538c9f308808b37073cda2"} Aug 04 20:42:08.737851 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:08.737854 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" event={"ID":"d3c033e8-fff1-425a-9166-632197c56a76","Type":"ContainerStarted","Data":"7da7d15f34c0f537f392f692c1ba773b564cfcb7c4fd3c96dd20e06df4cdd629"} Aug 04 20:42:08.738081 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:08.737866 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" event={"ID":"d3c033e8-fff1-425a-9166-632197c56a76","Type":"ContainerStarted","Data":"8350af2301ad0a63353984d0a45e2fa56c86a5ea22a24ca8f1963414a1df4f46"} Aug 04 20:42:08.760226 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:08.760179 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" podStartSLOduration=1.7885433750000002 podStartE2EDuration="5.76016404s" totalImagesPullingTime="3.971620665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:08.759182079 +0000 UTC m=+62.906052633" watchObservedRunningTime="2026-08-04 20:42:08.76016404 +0000 UTC m=+62.907034628" Aug 04 20:42:09.651457 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:09.651423 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:09.651808 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:09.651473 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:09.651808 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:09.651542 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 20:42:09.651808 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:09.651582 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 20:42:09.651808 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:09.651628 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:17.651608463 +0000 UTC m=+71.798479037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : configmap "kube-rbac-proxy" not found Aug 04 20:42:09.651808 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:09.651645 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:17.651637392 +0000 UTC m=+71.798507925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : secret "insights-runtime-extractor-tls" not found Aug 04 20:42:11.061653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.061617 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:11.062106 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.061665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:42:11.062106 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.061691 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:42:11.062106 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:11.061763 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 20:42:11.062106 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.061764 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:42:11.062106 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:11.061843 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls podName:f7ea2c4d-1554-402c-b432-002d85830ce9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:19.061821708 +0000 UTC m=+73.208692263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-nrj2z" (UID: "f7ea2c4d-1554-402c-b432-002d85830ce9") : secret "samples-operator-tls" not found Aug 04 20:42:11.064921 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.064887 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 20:42:11.065031 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.065012 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 20:42:11.065089 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.065044 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 20:42:11.072582 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:11.072563 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Aug 04 20:42:11.072641 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:11.072630 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs podName:e6116bd3-50d9-4d61-95c1-ea0cc53b88ab nodeName:}" failed. No retries permitted until 2026-08-04 20:43:15.072614833 +0000 UTC m=+129.219485383 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs") pod "network-metrics-daemon-qhmdn" (UID: "e6116bd3-50d9-4d61-95c1-ea0cc53b88ab") : secret "metrics-daemon-secret" not found Aug 04 20:42:11.074892 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.074862 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a43ca81a-b9a5-45da-b3fa-09452b3296f6-original-pull-secret\") pod \"global-pull-secret-syncer-6j5xp\" (UID: \"a43ca81a-b9a5-45da-b3fa-09452b3296f6\") " pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:42:11.075252 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.075237 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 20:42:11.085815 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.085793 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8wn6\" (UniqueName: \"kubernetes.io/projected/47981c6e-01ce-496b-8d23-7c1c795cdbc6-kube-api-access-d8wn6\") pod \"network-check-target-lzgsl\" (UID: \"47981c6e-01ce-496b-8d23-7c1c795cdbc6\") " pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:42:11.265788 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.265748 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6j5xp" Aug 04 20:42:11.275671 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.275643 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nn596\"" Aug 04 20:42:11.281984 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.281961 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:42:11.392545 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.392512 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6j5xp"] Aug 04 20:42:11.395260 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:11.395233 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43ca81a_b9a5_45da_b3fa_09452b3296f6.slice/crio-9541d3adacb5a3198885bc967b075c4586b21271b3e33dd4e31f28499ac5207b WatchSource:0}: Error finding container 9541d3adacb5a3198885bc967b075c4586b21271b3e33dd4e31f28499ac5207b: Status 404 returned error can't find the container with id 9541d3adacb5a3198885bc967b075c4586b21271b3e33dd4e31f28499ac5207b Aug 04 20:42:11.409875 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.409852 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lzgsl"] Aug 04 20:42:11.412346 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:11.412325 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47981c6e_01ce_496b_8d23_7c1c795cdbc6.slice/crio-3dcc3d4b424f40261338311b83e9bc12b66e32962c64e57bb0157240ff877e1d WatchSource:0}: Error finding container 3dcc3d4b424f40261338311b83e9bc12b66e32962c64e57bb0157240ff877e1d: Status 404 returned error can't find the container with id 3dcc3d4b424f40261338311b83e9bc12b66e32962c64e57bb0157240ff877e1d Aug 04 20:42:11.747365 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.747328 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6j5xp" event={"ID":"a43ca81a-b9a5-45da-b3fa-09452b3296f6","Type":"ContainerStarted","Data":"9541d3adacb5a3198885bc967b075c4586b21271b3e33dd4e31f28499ac5207b"} Aug 04 20:42:11.748432 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:11.748410 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lzgsl" event={"ID":"47981c6e-01ce-496b-8d23-7c1c795cdbc6","Type":"ContainerStarted","Data":"3dcc3d4b424f40261338311b83e9bc12b66e32962c64e57bb0157240ff877e1d"} Aug 04 20:42:12.297177 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:12.297143 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:12.403333 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:12.403289 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:12.728304 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:12.728268 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:12.838218 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:12.838181 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:12.928112 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:12.928077 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:13.031456 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:13.031375 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:13.078541 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:13.078507 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:42:13.078728 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:13.078681 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 20:42:13.078785 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:13.078770 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls podName:cd2bb155-816d-4b33-9c0f-528c36d56f55 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:45.078739408 +0000 UTC m=+99.225609944 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls") pod "dns-default-mn9mw" (UID: "cd2bb155-816d-4b33-9c0f-528c36d56f55") : secret "dns-default-metrics-tls" not found Aug 04 20:42:13.135217 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:13.135170 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:14.635357 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:14.635322 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:14.700503 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:14.700462 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:15.093914 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:15.093872 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:15.553167 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:15.553081 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:16.666321 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:16.666287 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:16.767029 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:16.766993 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:16.768126 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:16.768093 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lzgsl" event={"ID":"47981c6e-01ce-496b-8d23-7c1c795cdbc6","Type":"ContainerStarted","Data":"fd99f19ae2e62e15d489824159ab9c62ae439ea70165369f4fe6cedd6ff45bea"} Aug 04 20:42:16.768267 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:16.768170 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:42:16.769531 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:16.769509 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6j5xp" event={"ID":"a43ca81a-b9a5-45da-b3fa-09452b3296f6","Type":"ContainerStarted","Data":"ad87b438428de981e57995410fc75f0dad513a2df83d086a73e9ddb621f2f898"} Aug 04 20:42:16.790178 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:16.790133 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lzgsl" podStartSLOduration=64.995018481 podStartE2EDuration="1m9.790120452s" totalImagesPullingTime="4.795101971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:16.78946215 +0000 UTC m=+70.936332706" watchObservedRunningTime="2026-08-04 20:42:16.790120452 +0000 UTC m=+70.936991006" Aug 04 20:42:16.805939 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:16.805882 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6j5xp" podStartSLOduration=64.981806999 podStartE2EDuration="1m9.805870136s" totalImagesPullingTime="4.824063137s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:16.805742844 +0000 UTC m=+70.952613399" watchObservedRunningTime="2026-08-04 20:42:16.805870136 +0000 UTC m=+70.952740690" Aug 04 20:42:16.868721 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:16.868651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:16.970147 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:16.970011 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:17.082298 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:17.082245 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:17.719304 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:17.719267 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:17.719679 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:17.719318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:17.719679 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:17.719486 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 20:42:17.719679 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:17.719551 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls podName:d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c nodeName:}" failed. No retries permitted until 2026-08-04 20:42:33.7195363 +0000 UTC m=+87.866406833 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27h2f" (UID: "d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c") : secret "insights-runtime-extractor-tls" not found Aug 04 20:42:17.719804 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:17.719786 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:18.177073 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:18.177036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:18.371097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:18.371061 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:18.579807 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:18.579728 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:18.670672 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:18.670637 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:18.763655 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:18.763619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:18.970327 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:18.970296 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:19.129651 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:19.129600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:19.132280 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:19.132259 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7ea2c4d-1554-402c-b432-002d85830ce9-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-nrj2z\" (UID: \"f7ea2c4d-1554-402c-b432-002d85830ce9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:19.231717 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:19.231646 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gsf5n\"" Aug 04 20:42:19.239783 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:19.239763 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" Aug 04 20:42:19.363583 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:19.363547 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z"] Aug 04 20:42:19.704026 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:19.703995 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:19.778429 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:19.778394 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" event={"ID":"f7ea2c4d-1554-402c-b432-002d85830ce9","Type":"ContainerStarted","Data":"0a560c981010690dd8260c8f1c4f12c78647c70f3d97ca7eb9e863b05d8ba872"} Aug 04 20:42:20.167591 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:20.167552 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:20.370576 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:20.370534 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:20.563804 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:20.563722 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:20.623708 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:20.623667 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:20.766162 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:20.766130 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:20.970725 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:20.970699 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:21.787155 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:21.787122 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" event={"ID":"f7ea2c4d-1554-402c-b432-002d85830ce9","Type":"ContainerStarted","Data":"31d307e9df1541c02d41763d9b5eeb48318f5d6deb9919c49988e2365748ce94"} Aug 04 20:42:21.787326 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:21.787173 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" event={"ID":"f7ea2c4d-1554-402c-b432-002d85830ce9","Type":"ContainerStarted","Data":"1c9c88aac90fe247719d368c1591549675aca0868301d242812c14935ee90ad7"} Aug 04 20:42:21.806398 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:21.806349 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-nrj2z" podStartSLOduration=17.429730951 podStartE2EDuration="18.806329969s" totalImagesPullingTime="1.376599018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:21.805957649 +0000 UTC m=+75.952828204" watchObservedRunningTime="2026-08-04 20:42:21.806329969 +0000 UTC m=+75.953200526" Aug 04 20:42:22.163809 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:22.163779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:22.368561 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:22.368525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:22.562582 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:22.562496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:22.689406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:22.689372 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:22.788988 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:22.788963 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:22.961489 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:22.961453 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:23.680943 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:23.680897 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:23.782691 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:23.782663 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:23.964641 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:23.964565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:24.366134 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:24.366100 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:24.571415 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:24.571371 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:24.767576 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:24.767494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:24.964504 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:24.964468 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:25.976686 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:25.976655 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:26.375401 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:26.375371 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:26.581755 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:26.581723 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:26.766503 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:26.766418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:26.930017 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:26.929985 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:27.154309 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:27.154275 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:27.256813 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:27.256784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:27.317118 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:27.317092 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:27.462631 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:27.462550 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:27.603458 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:27.603423 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:27.700691 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:27.700658 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:27.843139 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:27.843082 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:28.037842 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:28.037807 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:28.102183 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:28.102100 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:28.441386 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:28.441308 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:28.500702 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:28.500670 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:28.652309 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:28.652278 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:28.753846 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:28.753760 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:29.067883 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:29.067802 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:29.673649 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:29.673620 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:32.047502 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:32.047468 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:32.148418 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:32.148386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:32.244139 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:32.244099 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:32.892543 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:32.892511 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:33.752523 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:33.752482 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:33.754982 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:33.754964 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27h2f\" (UID: \"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c\") " pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:34.032396 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:34.032316 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d7shv\"" Aug 04 20:42:34.040550 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:34.040534 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-27h2f" Aug 04 20:42:34.168311 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:34.168276 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-27h2f"] Aug 04 20:42:34.170920 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:34.170880 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f8d3e6_cbdf_4e8d_a955_1a1a3489953c.slice/crio-321b46f0443b4ec4dad39ec7861c74c57ef8df84f66204f2629efcf88931f379 WatchSource:0}: Error finding container 321b46f0443b4ec4dad39ec7861c74c57ef8df84f66204f2629efcf88931f379: Status 404 returned error can't find the container with id 321b46f0443b4ec4dad39ec7861c74c57ef8df84f66204f2629efcf88931f379 Aug 04 20:42:34.831757 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:34.831726 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27h2f" event={"ID":"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c","Type":"ContainerStarted","Data":"5c782762fc45f59853728b9073f94ef87f1bfd2a6c75c700a19bc9f7c233a9f2"} Aug 04 20:42:34.832151 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:34.831779 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27h2f" event={"ID":"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c","Type":"ContainerStarted","Data":"ec88833920814fd68706e280f90ef5fe98928ea6474255ec491df2e42d646773"} Aug 04 20:42:34.832151 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:34.831798 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27h2f" event={"ID":"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c","Type":"ContainerStarted","Data":"321b46f0443b4ec4dad39ec7861c74c57ef8df84f66204f2629efcf88931f379"} Aug 04 20:42:35.270545 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:35.270516 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:35.578746 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:35.578641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:35.684305 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:35.684270 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:36.839503 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:36.839464 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27h2f" event={"ID":"d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c","Type":"ContainerStarted","Data":"111db24680d14c686593bf66fddbe182c4d8e640d2169e25275c4283e507eda9"} Aug 04 20:42:36.865326 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:36.865277 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-27h2f" podStartSLOduration=34.036052138 podStartE2EDuration="35.86526259s" totalImagesPullingTime="1.829210452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:42:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:36.865018815 +0000 UTC m=+91.011889395" watchObservedRunningTime="2026-08-04 20:42:36.86526259 +0000 UTC m=+91.012133144" Aug 04 20:42:37.676724 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:37.676687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:37.746223 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:37.746195 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:38.706898 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.706866 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-775b9ddbf6-w9nsl"] Aug 04 20:42:38.732450 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.732423 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:38.749620 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.749591 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-775b9ddbf6-w9nsl"] Aug 04 20:42:38.749765 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.749683 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.752977 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.752955 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Aug 04 20:42:38.753170 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.753148 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Aug 04 20:42:38.753318 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.753298 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Aug 04 20:42:38.753412 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.753392 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5m6zq\"" Aug 04 20:42:38.753412 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.753403 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 04 20:42:38.753749 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.753732 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Aug 04 20:42:38.755042 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.754947 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 04 20:42:38.755042 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.755035 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Aug 04 20:42:38.765960 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.765930 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Aug 04 20:42:38.830119 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.830083 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:38.890726 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.890695 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-config\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.890726 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.890726 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-oauth-serving-cert\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.890899 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.890757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-trusted-ca-bundle\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.890899 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.890778 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npzdc\" (UniqueName: \"kubernetes.io/projected/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-kube-api-access-npzdc\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.890899 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.890826 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-service-ca\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.890899 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.890849 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-oauth-config\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.890899 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.890888 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-serving-cert\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.937089 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.937062 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:38.991721 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.991655 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-config\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.991721 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.991686 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-oauth-serving-cert\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.991929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.991722 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-trusted-ca-bundle\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.991929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.991746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npzdc\" (UniqueName: \"kubernetes.io/projected/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-kube-api-access-npzdc\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.991929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.991770 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-service-ca\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.991929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.991802 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-oauth-config\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.991929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.991847 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-serving-cert\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.992576 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.992484 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-config\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.992576 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.992576 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-service-ca\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.992802 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.992643 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-trusted-ca-bundle\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.992863 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.992848 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-oauth-serving-cert\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.994360 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.994340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-oauth-config\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:38.994466 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:38.994449 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-serving-cert\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:39.022196 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:39.022168 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npzdc\" (UniqueName: \"kubernetes.io/projected/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-kube-api-access-npzdc\") pod \"console-775b9ddbf6-w9nsl\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:39.059058 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:39.059034 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:39.226468 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:39.226036 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-775b9ddbf6-w9nsl"] Aug 04 20:42:39.232041 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:39.232013 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e81c5fe_7ce2_46ca_8e33_882671cfdc46.slice/crio-b0708a7f916ce8b87dc6d37d3527858912b578051dbfd1d095baeecd47b6b318 WatchSource:0}: Error finding container b0708a7f916ce8b87dc6d37d3527858912b578051dbfd1d095baeecd47b6b318: Status 404 returned error can't find the container with id b0708a7f916ce8b87dc6d37d3527858912b578051dbfd1d095baeecd47b6b318 Aug 04 20:42:39.850789 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:39.850754 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775b9ddbf6-w9nsl" event={"ID":"2e81c5fe-7ce2-46ca-8e33-882671cfdc46","Type":"ContainerStarted","Data":"b0708a7f916ce8b87dc6d37d3527858912b578051dbfd1d095baeecd47b6b318"} Aug 04 20:42:41.499340 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:41.499307 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:41.569344 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:41.569312 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:42.242210 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.242182 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:42.284797 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.284778 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-7zq46"] Aug 04 20:42:42.287604 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.287587 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.290363 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.290346 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 20:42:42.291374 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.291355 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Aug 04 20:42:42.291374 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.291365 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-twdv7\"" Aug 04 20:42:42.291541 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.291385 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 20:42:42.291541 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.291440 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 20:42:42.291541 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.291386 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Aug 04 20:42:42.299112 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.299093 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-7zq46"] Aug 04 20:42:42.322469 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.322440 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:42.422404 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.422368 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgmcj\" (UniqueName: \"kubernetes.io/projected/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-kube-api-access-sgmcj\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.422563 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.422428 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.422563 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.422500 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.422563 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.422550 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.443348 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.443323 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:42.523277 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.523214 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.523277 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.523258 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgmcj\" (UniqueName: \"kubernetes.io/projected/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-kube-api-access-sgmcj\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.523616 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.523304 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.523616 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.523336 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.524054 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.524036 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.525851 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.525830 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.525972 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.525831 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.531171 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.531148 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgmcj\" (UniqueName: \"kubernetes.io/projected/9f3871a9-49cd-45ef-8c22-940f2fbbe7f3-kube-api-access-sgmcj\") pod \"prometheus-operator-758d7cb874-7zq46\" (UID: \"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.596990 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.596965 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" Aug 04 20:42:42.725648 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.725619 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-7zq46"] Aug 04 20:42:42.728575 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:42.728552 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f3871a9_49cd_45ef_8c22_940f2fbbe7f3.slice/crio-d41b8c343b21c0f9bbd4cb9ca5f59accb097479222a15551c079687919b5073b WatchSource:0}: Error finding container d41b8c343b21c0f9bbd4cb9ca5f59accb097479222a15551c079687919b5073b: Status 404 returned error can't find the container with id d41b8c343b21c0f9bbd4cb9ca5f59accb097479222a15551c079687919b5073b Aug 04 20:42:42.862794 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.862765 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775b9ddbf6-w9nsl" event={"ID":"2e81c5fe-7ce2-46ca-8e33-882671cfdc46","Type":"ContainerStarted","Data":"02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed"} Aug 04 20:42:42.864038 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.864014 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" event={"ID":"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3","Type":"ContainerStarted","Data":"d41b8c343b21c0f9bbd4cb9ca5f59accb097479222a15551c079687919b5073b"} Aug 04 20:42:42.881593 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:42.881538 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-775b9ddbf6-w9nsl" podStartSLOduration=1.873234826 podStartE2EDuration="4.881524612s" totalImagesPullingTime="3.008289786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:42.880793466 +0000 UTC m=+97.027664013" watchObservedRunningTime="2026-08-04 20:42:42.881524612 +0000 UTC m=+97.028395166" Aug 04 20:42:43.869539 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:43.869511 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" event={"ID":"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3","Type":"ContainerStarted","Data":"d1efd72fc0d6cbc8b66ad2eaf9ac689b9d24b9133587c884f295ff63513e9f78"} Aug 04 20:42:44.037676 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:44.037576 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:44.479653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:44.479615 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:44.541567 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:44.541538 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:44.874239 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:44.874206 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" event={"ID":"9f3871a9-49cd-45ef-8c22-940f2fbbe7f3","Type":"ContainerStarted","Data":"3a56246c5b1b19e38ed900f0ee7641b264d7f54927ab145d6ddd36a421fdf589"} Aug 04 20:42:44.895023 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:44.894971 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-7zq46" podStartSLOduration=1.86359766 podStartE2EDuration="2.894956342s" totalImagesPullingTime="1.031358682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:44.893640661 +0000 UTC m=+99.040511208" watchObservedRunningTime="2026-08-04 20:42:44.894956342 +0000 UTC m=+99.041826888" Aug 04 20:42:45.144077 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:45.143984 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:42:45.146451 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:45.146424 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd2bb155-816d-4b33-9c0f-528c36d56f55-metrics-tls\") pod \"dns-default-mn9mw\" (UID: \"cd2bb155-816d-4b33-9c0f-528c36d56f55\") " pod="openshift-dns/dns-default-mn9mw" Aug 04 20:42:45.167290 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:45.167268 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kcgl6\"" Aug 04 20:42:45.175301 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:45.175285 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mn9mw" Aug 04 20:42:45.295405 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:45.295383 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mn9mw"] Aug 04 20:42:45.297557 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:45.297537 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2bb155_816d_4b33_9c0f_528c36d56f55.slice/crio-36f7b3014a832e8316bd857ec5bbd469e21ea53013f89b2240ba8cc02d90616b WatchSource:0}: Error finding container 36f7b3014a832e8316bd857ec5bbd469e21ea53013f89b2240ba8cc02d90616b: Status 404 returned error can't find the container with id 36f7b3014a832e8316bd857ec5bbd469e21ea53013f89b2240ba8cc02d90616b Aug 04 20:42:45.879105 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:45.879070 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mn9mw" event={"ID":"cd2bb155-816d-4b33-9c0f-528c36d56f55","Type":"ContainerStarted","Data":"36f7b3014a832e8316bd857ec5bbd469e21ea53013f89b2240ba8cc02d90616b"} Aug 04 20:42:46.400024 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.399992 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:46.507559 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.507532 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:46.625627 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.625595 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:46.657638 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.657566 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj"] Aug 04 20:42:46.673225 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.673194 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.676423 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.676400 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Aug 04 20:42:46.676551 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.676459 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nl9w7\"" Aug 04 20:42:46.676765 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.676746 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Aug 04 20:42:46.688638 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.688615 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj"] Aug 04 20:42:46.716365 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.716338 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x5mp5"] Aug 04 20:42:46.737321 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.737298 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.740304 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.740280 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 20:42:46.740533 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.740511 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 20:42:46.740640 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.740614 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 20:42:46.740710 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.740659 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cjqnv\"" Aug 04 20:42:46.740771 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.740722 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:46.756618 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756592 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/401f7af5-f9af-49cd-9d62-106877917d06-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.756737 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756634 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-tls\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.756737 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756659 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/401f7af5-f9af-49cd-9d62-106877917d06-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.756832 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756734 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b416b2ad-ae78-4a40-a731-5b867e854a9d-root\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.756832 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b416b2ad-ae78-4a40-a731-5b867e854a9d-sys\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.756832 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.756832 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756817 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b416b2ad-ae78-4a40-a731-5b867e854a9d-metrics-client-ca\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.757035 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/401f7af5-f9af-49cd-9d62-106877917d06-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.757035 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756898 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clnlh\" (UniqueName: \"kubernetes.io/projected/b416b2ad-ae78-4a40-a731-5b867e854a9d-kube-api-access-clnlh\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.757035 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756954 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.757035 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.756991 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-textfile\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.757035 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.757007 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9mr4\" (UniqueName: \"kubernetes.io/projected/401f7af5-f9af-49cd-9d62-106877917d06-kube-api-access-r9mr4\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.757250 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.757060 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-wtmp\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.829712 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.829681 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:46.857774 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.857738 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b416b2ad-ae78-4a40-a731-5b867e854a9d-metrics-client-ca\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.857957 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.857788 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/401f7af5-f9af-49cd-9d62-106877917d06-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.857957 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.857815 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clnlh\" (UniqueName: \"kubernetes.io/projected/b416b2ad-ae78-4a40-a731-5b867e854a9d-kube-api-access-clnlh\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.857957 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.857848 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.857957 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.857890 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-textfile\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.857957 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.857924 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9mr4\" (UniqueName: \"kubernetes.io/projected/401f7af5-f9af-49cd-9d62-106877917d06-kube-api-access-r9mr4\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.857957 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.857951 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-wtmp\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.858275 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.857978 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/401f7af5-f9af-49cd-9d62-106877917d06-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.858275 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.857996 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-tls\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.858275 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.858019 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/401f7af5-f9af-49cd-9d62-106877917d06-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.858275 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.858051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b416b2ad-ae78-4a40-a731-5b867e854a9d-root\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.858275 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.858078 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b416b2ad-ae78-4a40-a731-5b867e854a9d-sys\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.858275 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.858104 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.859035 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.859007 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-wtmp\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.860744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.859320 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b416b2ad-ae78-4a40-a731-5b867e854a9d-metrics-client-ca\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.860744 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:46.859842 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Aug 04 20:42:46.860744 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:42:46.859926 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-tls podName:b416b2ad-ae78-4a40-a731-5b867e854a9d nodeName:}" failed. No retries permitted until 2026-08-04 20:42:47.359887914 +0000 UTC m=+101.506758471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-tls") pod "node-exporter-x5mp5" (UID: "b416b2ad-ae78-4a40-a731-5b867e854a9d") : secret "node-exporter-tls" not found Aug 04 20:42:46.860744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.860050 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.860744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.860309 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b416b2ad-ae78-4a40-a731-5b867e854a9d-sys\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.860744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.860360 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b416b2ad-ae78-4a40-a731-5b867e854a9d-root\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.860744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.860587 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-textfile\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.861123 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.860769 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/401f7af5-f9af-49cd-9d62-106877917d06-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.861433 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.861404 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.861838 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.861813 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/401f7af5-f9af-49cd-9d62-106877917d06-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.863167 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.863141 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/401f7af5-f9af-49cd-9d62-106877917d06-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.868774 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.868740 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clnlh\" (UniqueName: \"kubernetes.io/projected/b416b2ad-ae78-4a40-a731-5b867e854a9d-kube-api-access-clnlh\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:46.869149 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.869126 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9mr4\" (UniqueName: \"kubernetes.io/projected/401f7af5-f9af-49cd-9d62-106877917d06-kube-api-access-r9mr4\") pod \"openshift-state-metrics-6bb4d7b4dd-7p6bj\" (UID: \"401f7af5-f9af-49cd-9d62-106877917d06\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:46.988837 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:46.988756 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" Aug 04 20:42:47.064371 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.064337 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:47.163922 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.163874 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:47.223440 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.223416 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj"] Aug 04 20:42:47.225994 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:47.225968 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod401f7af5_f9af_49cd_9d62_106877917d06.slice/crio-c83f80b0407404b4f54b35d100e32e728591e7c3ddf258530c939de47977de4c WatchSource:0}: Error finding container c83f80b0407404b4f54b35d100e32e728591e7c3ddf258530c939de47977de4c: Status 404 returned error can't find the container with id c83f80b0407404b4f54b35d100e32e728591e7c3ddf258530c939de47977de4c Aug 04 20:42:47.264146 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.264123 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:47.362871 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.362837 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-tls\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:47.365852 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.365820 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b416b2ad-ae78-4a40-a731-5b867e854a9d-node-exporter-tls\") pod \"node-exporter-x5mp5\" (UID: \"b416b2ad-ae78-4a40-a731-5b867e854a9d\") " pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:47.393696 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.393647 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:47.647745 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.647717 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x5mp5" Aug 04 20:42:47.664863 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.664788 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:47.675230 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:47.675198 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb416b2ad_ae78_4a40_a731_5b867e854a9d.slice/crio-d88e3179fccda5a22f0a8f69aa4953af115c219344f265c9c82c082a7a811a47 WatchSource:0}: Error finding container d88e3179fccda5a22f0a8f69aa4953af115c219344f265c9c82c082a7a811a47: Status 404 returned error can't find the container with id d88e3179fccda5a22f0a8f69aa4953af115c219344f265c9c82c082a7a811a47 Aug 04 20:42:47.756106 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.756075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:47.775591 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.775515 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lzgsl" Aug 04 20:42:47.887813 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.887777 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mn9mw" event={"ID":"cd2bb155-816d-4b33-9c0f-528c36d56f55","Type":"ContainerStarted","Data":"aaede3ef1cfbb2b87556d53a721568864d1ada74492023c233211113642110ed"} Aug 04 20:42:47.888001 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.887832 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mn9mw" event={"ID":"cd2bb155-816d-4b33-9c0f-528c36d56f55","Type":"ContainerStarted","Data":"d844996ca7277737e2f63d7f09d3813ae1e98d7e49b84f16bdb057ba7a3ecd97"} Aug 04 20:42:47.888001 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.887872 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mn9mw" Aug 04 20:42:47.889687 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.889662 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" event={"ID":"401f7af5-f9af-49cd-9d62-106877917d06","Type":"ContainerStarted","Data":"3e24bbd658360b89f4dad6cf2c38d8def2c5c6be4945152e10403636a8713325"} Aug 04 20:42:47.889814 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.889708 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" event={"ID":"401f7af5-f9af-49cd-9d62-106877917d06","Type":"ContainerStarted","Data":"4236a7c1d367cb22bbd1d867707ce7e1f009c03e41346d87260cf1b0e0fa7625"} Aug 04 20:42:47.889814 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.889734 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" event={"ID":"401f7af5-f9af-49cd-9d62-106877917d06","Type":"ContainerStarted","Data":"c83f80b0407404b4f54b35d100e32e728591e7c3ddf258530c939de47977de4c"} Aug 04 20:42:47.890683 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.890663 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x5mp5" event={"ID":"b416b2ad-ae78-4a40-a731-5b867e854a9d","Type":"ContainerStarted","Data":"d88e3179fccda5a22f0a8f69aa4953af115c219344f265c9c82c082a7a811a47"} Aug 04 20:42:47.905050 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:47.905008 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mn9mw" podStartSLOduration=64.740364791 podStartE2EDuration="1m6.904992351s" totalImagesPullingTime="2.16462756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:47.904252358 +0000 UTC m=+102.051122904" watchObservedRunningTime="2026-08-04 20:42:47.904992351 +0000 UTC m=+102.051862913" Aug 04 20:42:48.289581 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.289477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:48.663654 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.663615 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b78876469-2ppjb"] Aug 04 20:42:48.678548 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.678520 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.681070 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.681037 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Aug 04 20:42:48.681212 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.681129 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Aug 04 20:42:48.681212 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.681142 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-59vbt\"" Aug 04 20:42:48.681212 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.681056 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bmnblmcl5pjpi\"" Aug 04 20:42:48.681212 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.681190 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Aug 04 20:42:48.681413 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.681147 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Aug 04 20:42:48.681723 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.681702 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Aug 04 20:42:48.686495 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.686452 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b78876469-2ppjb"] Aug 04 20:42:48.689562 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.689522 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:48.755818 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.755788 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:48.772336 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.772308 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.772421 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.772348 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/494573a3-8394-4adb-99a8-005042b3de04-metrics-client-ca\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.772421 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.772385 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-tls\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.772421 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.772407 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.772529 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.772433 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-grpc-tls\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.772529 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.772491 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bt2n\" (UniqueName: \"kubernetes.io/projected/494573a3-8394-4adb-99a8-005042b3de04-kube-api-access-5bt2n\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.772529 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.772525 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.772620 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.772558 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.873075 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.873037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/494573a3-8394-4adb-99a8-005042b3de04-metrics-client-ca\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.873254 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.873091 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-tls\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.873254 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.873113 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.873254 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.873136 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-grpc-tls\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.873254 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.873164 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bt2n\" (UniqueName: \"kubernetes.io/projected/494573a3-8394-4adb-99a8-005042b3de04-kube-api-access-5bt2n\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.873254 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.873207 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.873254 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.873241 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.873551 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.873300 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.874178 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.874148 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/494573a3-8394-4adb-99a8-005042b3de04-metrics-client-ca\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.877281 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.876750 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.877281 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.877211 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.877484 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.877458 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.877573 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.877517 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-grpc-tls\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.877790 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.877769 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-tls\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.878427 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.878402 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/494573a3-8394-4adb-99a8-005042b3de04-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.882465 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.882438 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bt2n\" (UniqueName: \"kubernetes.io/projected/494573a3-8394-4adb-99a8-005042b3de04-kube-api-access-5bt2n\") pod \"thanos-querier-b78876469-2ppjb\" (UID: \"494573a3-8394-4adb-99a8-005042b3de04\") " pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:48.991637 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:48.991533 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:49.060206 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.060174 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:49.060352 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.060264 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:49.064836 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.064816 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:49.492186 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.492145 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b78876469-2ppjb"] Aug 04 20:42:49.494853 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:49.494821 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494573a3_8394_4adb_99a8_005042b3de04.slice/crio-2e630f1220c9ac5bacb8979ed6d07fa62291a5ab715680f486be127b5cdbc54d WatchSource:0}: Error finding container 2e630f1220c9ac5bacb8979ed6d07fa62291a5ab715680f486be127b5cdbc54d: Status 404 returned error can't find the container with id 2e630f1220c9ac5bacb8979ed6d07fa62291a5ab715680f486be127b5cdbc54d Aug 04 20:42:49.606408 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.606324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:49.691591 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.691555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:49.815786 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.815744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:49.902389 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.902352 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" event={"ID":"401f7af5-f9af-49cd-9d62-106877917d06","Type":"ContainerStarted","Data":"930deea8254ddb233e82397fc05d713d7f8eb70e07a3a52a0a17f5255ac5b626"} Aug 04 20:42:49.904213 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.904183 2576 generic.go:356] "Generic (PLEG): container finished" podID="b416b2ad-ae78-4a40-a731-5b867e854a9d" containerID="996a86d55a4bfa85f7a44ba56a0ba0a63f22515d35e8ceae8bc1bcf22a823833" exitCode=0 Aug 04 20:42:49.904334 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.904263 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x5mp5" event={"ID":"b416b2ad-ae78-4a40-a731-5b867e854a9d","Type":"ContainerDied","Data":"996a86d55a4bfa85f7a44ba56a0ba0a63f22515d35e8ceae8bc1bcf22a823833"} Aug 04 20:42:49.905827 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.905799 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" event={"ID":"494573a3-8394-4adb-99a8-005042b3de04","Type":"ContainerStarted","Data":"2e630f1220c9ac5bacb8979ed6d07fa62291a5ab715680f486be127b5cdbc54d"} Aug 04 20:42:49.910410 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.910383 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:42:49.922973 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:49.922896 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-7p6bj" podStartSLOduration=2.178948017 podStartE2EDuration="3.922879758s" totalImagesPullingTime="1.743931741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:49.921733534 +0000 UTC m=+104.068604092" watchObservedRunningTime="2026-08-04 20:42:49.922879758 +0000 UTC m=+104.069750313" Aug 04 20:42:50.384112 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:50.384074 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:50.912117 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:50.912075 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x5mp5" event={"ID":"b416b2ad-ae78-4a40-a731-5b867e854a9d","Type":"ContainerStarted","Data":"1d24316234e2e19fec7df4ad8eb11dd9cb7b5441a59504e2e682a5710fd34046"} Aug 04 20:42:50.912551 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:50.912139 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x5mp5" event={"ID":"b416b2ad-ae78-4a40-a731-5b867e854a9d","Type":"ContainerStarted","Data":"5660f26709cf617cdde088f56cae8e06514870f26be9a2153c67d02800315177"} Aug 04 20:42:50.935858 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:50.935800 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x5mp5" podStartSLOduration=3.185375775 podStartE2EDuration="4.935783505s" totalImagesPullingTime="1.674187417s" totalInitContainerRuntime="76.220313ms" isStatefulPod=true podCreationTimestamp="2026-08-04 20:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:50.934071347 +0000 UTC m=+105.080941905" watchObservedRunningTime="2026-08-04 20:42:50.935783505 +0000 UTC m=+105.082654060" Aug 04 20:42:51.423976 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:51.423876 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:51.488651 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:51.488621 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:51.770815 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:51.770724 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:51.919042 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:51.919006 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" event={"ID":"494573a3-8394-4adb-99a8-005042b3de04","Type":"ContainerStarted","Data":"3c995cfd7703f041a09c61cb491d61255aa46afae62e1f24bd8fc99972a32837"} Aug 04 20:42:51.919567 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:51.919127 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" event={"ID":"494573a3-8394-4adb-99a8-005042b3de04","Type":"ContainerStarted","Data":"c7187ae2b98bfbf066262817c869e9742bd931130721dbb6bd636500b6482317"} Aug 04 20:42:51.919567 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:51.919173 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" event={"ID":"494573a3-8394-4adb-99a8-005042b3de04","Type":"ContainerStarted","Data":"edeef5bd3e2ecb6f33bff3da9a3618fa1b3ac8576482f21c5b21e4465b31851c"} Aug 04 20:42:52.888999 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:52.888955 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:52.926259 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:52.926215 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" event={"ID":"494573a3-8394-4adb-99a8-005042b3de04","Type":"ContainerStarted","Data":"aea3aea0392d3f38ae87bee148069039d3669d3a20a930feae169666861b3326"} Aug 04 20:42:52.926741 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:52.926300 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" event={"ID":"494573a3-8394-4adb-99a8-005042b3de04","Type":"ContainerStarted","Data":"986350b1c9d6a3d5dce3127a782a865babbf9f30053f820437c866709ef57ac9"} Aug 04 20:42:52.926741 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:52.926340 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" event={"ID":"494573a3-8394-4adb-99a8-005042b3de04","Type":"ContainerStarted","Data":"ff2b585866b9ea457519c9d45c645e3edaed00865ea597e2b49460cfe3cb264d"} Aug 04 20:42:52.926741 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:52.926475 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:42:52.951809 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:52.951758 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" podStartSLOduration=2.5247685349999998 podStartE2EDuration="4.951741892s" totalImagesPullingTime="2.426973357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:52.95020793 +0000 UTC m=+107.097078504" watchObservedRunningTime="2026-08-04 20:42:52.951741892 +0000 UTC m=+107.098612447" Aug 04 20:42:53.748101 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:53.748069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:57.078489 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.078447 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cd8bfd658-rsrkj"] Aug 04 20:42:57.082642 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.082610 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.093255 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.093229 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cd8bfd658-rsrkj"] Aug 04 20:42:57.140323 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.140289 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-trusted-ca-bundle\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.140495 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.140331 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-oauth-serving-cert\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.140495 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.140351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-oauth-config\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.140495 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.140401 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-serving-cert\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.140495 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.140428 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-config\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.140495 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.140477 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5vhz\" (UniqueName: \"kubernetes.io/projected/0a2950ef-3b3c-426d-a1cb-705dab37ec52-kube-api-access-x5vhz\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.140495 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.140497 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-service-ca\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.241160 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.241113 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-trusted-ca-bundle\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.241362 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.241174 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-oauth-serving-cert\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.241362 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.241199 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-oauth-config\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.241362 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.241225 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-serving-cert\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.241362 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.241253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-config\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.241362 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.241294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5vhz\" (UniqueName: \"kubernetes.io/projected/0a2950ef-3b3c-426d-a1cb-705dab37ec52-kube-api-access-x5vhz\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.241602 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.241433 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-service-ca\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.247335 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.242569 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-trusted-ca-bundle\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.247335 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.242583 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-service-ca\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.247335 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.242711 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-config\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.247335 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.242726 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-oauth-serving-cert\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.247335 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.244594 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-oauth-config\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.247335 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.246473 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-serving-cert\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.252601 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.252575 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5vhz\" (UniqueName: \"kubernetes.io/projected/0a2950ef-3b3c-426d-a1cb-705dab37ec52-kube-api-access-x5vhz\") pod \"console-5cd8bfd658-rsrkj\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.393142 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.393104 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:42:57.523754 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.523718 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cd8bfd658-rsrkj"] Aug 04 20:42:57.527009 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:42:57.526975 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a2950ef_3b3c_426d_a1cb_705dab37ec52.slice/crio-28821ae2a3ee5cf2bf4c544a03830eb147089afc4c0c012648dbdb3a1451f8e7 WatchSource:0}: Error finding container 28821ae2a3ee5cf2bf4c544a03830eb147089afc4c0c012648dbdb3a1451f8e7: Status 404 returned error can't find the container with id 28821ae2a3ee5cf2bf4c544a03830eb147089afc4c0c012648dbdb3a1451f8e7 Aug 04 20:42:57.897057 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.897031 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mn9mw" Aug 04 20:42:57.950617 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.950578 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd8bfd658-rsrkj" event={"ID":"0a2950ef-3b3c-426d-a1cb-705dab37ec52","Type":"ContainerStarted","Data":"decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9"} Aug 04 20:42:57.950807 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.950633 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd8bfd658-rsrkj" event={"ID":"0a2950ef-3b3c-426d-a1cb-705dab37ec52","Type":"ContainerStarted","Data":"28821ae2a3ee5cf2bf4c544a03830eb147089afc4c0c012648dbdb3a1451f8e7"} Aug 04 20:42:57.967839 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:57.967793 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cd8bfd658-rsrkj" podStartSLOduration=0.967775128 podStartE2EDuration="967.775128ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:57.967529667 +0000 UTC m=+112.114400221" watchObservedRunningTime="2026-08-04 20:42:57.967775128 +0000 UTC m=+112.114645757" Aug 04 20:42:58.477066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:58.477023 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:58.539211 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:58.539174 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:42:58.936753 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:42:58.936726 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b78876469-2ppjb" Aug 04 20:43:00.767658 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:00.767622 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:00.831473 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:00.831397 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:00.905652 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:00.905617 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:00.962591 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:00.962545 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:01.141963 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:01.141882 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:01.211598 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:01.211565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:01.293134 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:01.293103 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:01.351852 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:01.351816 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:04.155088 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:04.155049 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:04.255499 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:04.255469 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:04.838303 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:04.838268 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:04.939296 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:04.939262 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:07.048325 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:07.048290 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:07.149076 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:07.149036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:07.393623 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:07.393590 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:43:07.393822 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:07.393660 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:43:07.398469 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:07.398443 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:43:07.995578 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:07.995545 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:43:08.060683 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:08.060649 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-775b9ddbf6-w9nsl"] Aug 04 20:43:08.184390 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:08.184308 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:08.285406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:08.285324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:08.383531 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:08.383502 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:08.484684 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:08.484651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:08.693665 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:08.693634 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:08.793771 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:08.793739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:13.221169 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:13.221125 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:13.280872 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:13.280837 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:15.090308 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:15.090275 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:43:15.092763 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:15.092731 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6116bd3-50d9-4d61-95c1-ea0cc53b88ab-metrics-certs\") pod \"network-metrics-daemon-qhmdn\" (UID: \"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab\") " pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:43:15.159377 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:15.159353 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8x4x\"" Aug 04 20:43:15.167502 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:15.167480 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmdn" Aug 04 20:43:15.295538 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:15.295515 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qhmdn"] Aug 04 20:43:15.297747 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:43:15.297718 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6116bd3_50d9_4d61_95c1_ea0cc53b88ab.slice/crio-6b715fb6e91af61f4ffdbb34fd1c80968ca8d6478412b8e627ab4a88b0bca227 WatchSource:0}: Error finding container 6b715fb6e91af61f4ffdbb34fd1c80968ca8d6478412b8e627ab4a88b0bca227: Status 404 returned error can't find the container with id 6b715fb6e91af61f4ffdbb34fd1c80968ca8d6478412b8e627ab4a88b0bca227 Aug 04 20:43:16.020950 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:16.020897 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qhmdn" event={"ID":"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab","Type":"ContainerStarted","Data":"6b715fb6e91af61f4ffdbb34fd1c80968ca8d6478412b8e627ab4a88b0bca227"} Aug 04 20:43:16.044524 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:16.044492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:16.145188 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:16.145147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:16.212537 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:16.212496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:17.026558 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:17.026522 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qhmdn" event={"ID":"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab","Type":"ContainerStarted","Data":"57395b4ff29f933a68cb74fbe0c82b967e2a94e1dba0bbce39edc3a42a54d4d7"} Aug 04 20:43:17.026721 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:17.026577 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qhmdn" event={"ID":"e6116bd3-50d9-4d61-95c1-ea0cc53b88ab","Type":"ContainerStarted","Data":"66e10446be6869b4bbc9d0e9e4c1ebc699c151d19588f79f3fa8863ea1ffc2ef"} Aug 04 20:43:17.075165 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:17.075119 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qhmdn" podStartSLOduration=129.072360325 podStartE2EDuration="2m10.075106214s" totalImagesPullingTime="1.002745889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:43:17.074355273 +0000 UTC m=+131.221225829" watchObservedRunningTime="2026-08-04 20:43:17.075106214 +0000 UTC m=+131.221976769" Aug 04 20:43:17.339078 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:17.339001 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:17.440870 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:17.440836 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:23.657161 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:23.657114 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" podUID="d3c033e8-fff1-425a-9166-632197c56a76" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Aug 04 20:43:23.714975 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:23.714944 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:23.779337 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:23.779313 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:30.686514 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:30.686478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:30.787357 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:30.787319 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:43:32.084484 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:32.084439 2576 generic.go:356] "Generic (PLEG): container finished" podID="8bf442c4-3348-46ae-ac9c-69b05f9dcbfb" containerID="13bde20f410650d7cd15d30809c446f61597cb3659c687a0da624fbc60079825" exitCode=0 Aug 04 20:43:32.084859 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:32.084514 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-6slt9" event={"ID":"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb","Type":"ContainerDied","Data":"13bde20f410650d7cd15d30809c446f61597cb3659c687a0da624fbc60079825"} Aug 04 20:43:32.084918 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:32.084867 2576 scope.go:117] "RemoveContainer" containerID="13bde20f410650d7cd15d30809c446f61597cb3659c687a0da624fbc60079825" Aug 04 20:43:33.085559 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.085491 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-775b9ddbf6-w9nsl" podUID="2e81c5fe-7ce2-46ca-8e33-882671cfdc46" containerName="console" containerID="cri-o://02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed" gracePeriod=15 Aug 04 20:43:33.090265 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.090231 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-6slt9" event={"ID":"8bf442c4-3348-46ae-ac9c-69b05f9dcbfb","Type":"ContainerStarted","Data":"926411c774bedba599597c60cf51829bdd629cd96af6f2666bac18bb8051fc85"} Aug 04 20:43:33.321518 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.321492 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:43:33.435720 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.435681 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-config\") pod \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " Aug 04 20:43:33.435934 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.435730 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-trusted-ca-bundle\") pod \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " Aug 04 20:43:33.435934 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.435794 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npzdc\" (UniqueName: \"kubernetes.io/projected/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-kube-api-access-npzdc\") pod \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " Aug 04 20:43:33.435934 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.435859 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-oauth-serving-cert\") pod \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " Aug 04 20:43:33.435934 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.435896 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-service-ca\") pod \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " Aug 04 20:43:33.436156 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.435965 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-oauth-config\") pod \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " Aug 04 20:43:33.436156 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.435989 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-serving-cert\") pod \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\" (UID: \"2e81c5fe-7ce2-46ca-8e33-882671cfdc46\") " Aug 04 20:43:33.436285 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.436265 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-config" (OuterVolumeSpecName: "console-config") pod "2e81c5fe-7ce2-46ca-8e33-882671cfdc46" (UID: "2e81c5fe-7ce2-46ca-8e33-882671cfdc46"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:43:33.436335 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.436289 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2e81c5fe-7ce2-46ca-8e33-882671cfdc46" (UID: "2e81c5fe-7ce2-46ca-8e33-882671cfdc46"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:43:33.436335 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.436306 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2e81c5fe-7ce2-46ca-8e33-882671cfdc46" (UID: "2e81c5fe-7ce2-46ca-8e33-882671cfdc46"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:43:33.436447 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.436416 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-service-ca" (OuterVolumeSpecName: "service-ca") pod "2e81c5fe-7ce2-46ca-8e33-882671cfdc46" (UID: "2e81c5fe-7ce2-46ca-8e33-882671cfdc46"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:43:33.438430 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.438401 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2e81c5fe-7ce2-46ca-8e33-882671cfdc46" (UID: "2e81c5fe-7ce2-46ca-8e33-882671cfdc46"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 20:43:33.438529 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.438441 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-kube-api-access-npzdc" (OuterVolumeSpecName: "kube-api-access-npzdc") pod "2e81c5fe-7ce2-46ca-8e33-882671cfdc46" (UID: "2e81c5fe-7ce2-46ca-8e33-882671cfdc46"). InnerVolumeSpecName "kube-api-access-npzdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 20:43:33.438529 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.438504 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2e81c5fe-7ce2-46ca-8e33-882671cfdc46" (UID: "2e81c5fe-7ce2-46ca-8e33-882671cfdc46"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 20:43:33.537299 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.537266 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npzdc\" (UniqueName: \"kubernetes.io/projected/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-kube-api-access-npzdc\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:43:33.537299 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.537292 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-oauth-serving-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:43:33.537299 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.537302 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:43:33.537497 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.537311 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-oauth-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:43:33.537497 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.537320 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-serving-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:43:33.537497 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.537327 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-console-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:43:33.537497 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.537335 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e81c5fe-7ce2-46ca-8e33-882671cfdc46-trusted-ca-bundle\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:43:33.656849 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:33.656817 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" podUID="d3c033e8-fff1-425a-9166-632197c56a76" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Aug 04 20:43:34.095265 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.095233 2576 generic.go:356] "Generic (PLEG): container finished" podID="2e81c5fe-7ce2-46ca-8e33-882671cfdc46" containerID="02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed" exitCode=0 Aug 04 20:43:34.095647 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.095278 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775b9ddbf6-w9nsl" Aug 04 20:43:34.095647 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.095279 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775b9ddbf6-w9nsl" event={"ID":"2e81c5fe-7ce2-46ca-8e33-882671cfdc46","Type":"ContainerDied","Data":"02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed"} Aug 04 20:43:34.095647 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.095409 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775b9ddbf6-w9nsl" event={"ID":"2e81c5fe-7ce2-46ca-8e33-882671cfdc46","Type":"ContainerDied","Data":"b0708a7f916ce8b87dc6d37d3527858912b578051dbfd1d095baeecd47b6b318"} Aug 04 20:43:34.095647 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.095460 2576 scope.go:117] "RemoveContainer" containerID="02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed" Aug 04 20:43:34.104522 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.104505 2576 scope.go:117] "RemoveContainer" containerID="02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed" Aug 04 20:43:34.104817 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:43:34.104791 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed\": container with ID starting with 02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed not found: ID does not exist" containerID="02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed" Aug 04 20:43:34.104897 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.104828 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed"} err="failed to get container status \"02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed\": rpc error: code = NotFound desc = could not find container \"02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed\": container with ID starting with 02abb36e59fda95067a204a65e9ceecd75912e080baa024107794a6b533b1fed not found: ID does not exist" Aug 04 20:43:34.115273 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.115251 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-775b9ddbf6-w9nsl"] Aug 04 20:43:34.118653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.118633 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-775b9ddbf6-w9nsl"] Aug 04 20:43:34.440785 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:34.440693 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e81c5fe-7ce2-46ca-8e33-882671cfdc46" path="/var/lib/kubelet/pods/2e81c5fe-7ce2-46ca-8e33-882671cfdc46/volumes" Aug 04 20:43:43.656983 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:43.656939 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" podUID="d3c033e8-fff1-425a-9166-632197c56a76" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Aug 04 20:43:43.657422 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:43.657017 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" Aug 04 20:43:43.657508 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:43.657490 2576 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"6abda1d2363e7a22b21c8fa8403a106dc2be25b01c538c9f308808b37073cda2"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" containerMessage="Container service-proxy failed liveness probe, will be restarted" Aug 04 20:43:43.657566 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:43.657524 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" podUID="d3c033e8-fff1-425a-9166-632197c56a76" containerName="service-proxy" containerID="cri-o://6abda1d2363e7a22b21c8fa8403a106dc2be25b01c538c9f308808b37073cda2" gracePeriod=30 Aug 04 20:43:44.137497 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:44.137449 2576 generic.go:356] "Generic (PLEG): container finished" podID="d3c033e8-fff1-425a-9166-632197c56a76" containerID="6abda1d2363e7a22b21c8fa8403a106dc2be25b01c538c9f308808b37073cda2" exitCode=2 Aug 04 20:43:44.137784 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:44.137512 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" event={"ID":"d3c033e8-fff1-425a-9166-632197c56a76","Type":"ContainerDied","Data":"6abda1d2363e7a22b21c8fa8403a106dc2be25b01c538c9f308808b37073cda2"} Aug 04 20:43:44.137784 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:43:44.137562 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcb8bc4c-nlb4t" event={"ID":"d3c033e8-fff1-425a-9166-632197c56a76","Type":"ContainerStarted","Data":"5bee7a9d913b73362ca7cce934e5d0b202be33e31a88b2a67db61aeafc9968e9"} Aug 04 20:44:05.878830 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:05.878729 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:06.080772 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:06.080726 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:13.935922 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:13.935881 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d79d5b4d-54j5t"] Aug 04 20:44:13.936282 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:13.936209 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e81c5fe-7ce2-46ca-8e33-882671cfdc46" containerName="console" Aug 04 20:44:13.936282 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:13.936223 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e81c5fe-7ce2-46ca-8e33-882671cfdc46" containerName="console" Aug 04 20:44:13.936282 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:13.936276 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e81c5fe-7ce2-46ca-8e33-882671cfdc46" containerName="console" Aug 04 20:44:13.940994 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:13.940969 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:13.955265 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:13.955238 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d79d5b4d-54j5t"] Aug 04 20:44:13.996365 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:13.996326 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:14.040684 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.040641 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-oauth-serving-cert\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.040840 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.040734 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-oauth-config\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.040840 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.040775 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-serving-cert\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.040840 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.040802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-service-ca\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.041018 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.040843 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npcrr\" (UniqueName: \"kubernetes.io/projected/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-kube-api-access-npcrr\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.041018 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.040877 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-config\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.041018 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.040928 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-trusted-ca-bundle\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.095356 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.095317 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:14.141605 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.141559 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-oauth-serving-cert\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.141808 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.141618 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-oauth-config\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.141808 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.141647 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-serving-cert\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.141808 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.141677 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-service-ca\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.141808 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.141699 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npcrr\" (UniqueName: \"kubernetes.io/projected/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-kube-api-access-npcrr\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.141808 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.141724 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-config\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.141808 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.141745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-trusted-ca-bundle\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.142415 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.142384 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-oauth-serving-cert\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.142578 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.142552 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-service-ca\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.142899 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.142882 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-config\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.143406 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.143384 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-trusted-ca-bundle\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.144620 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.144595 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-oauth-config\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.144620 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.144612 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-serving-cert\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.149793 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.149768 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npcrr\" (UniqueName: \"kubernetes.io/projected/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-kube-api-access-npcrr\") pod \"console-5d79d5b4d-54j5t\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.196394 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.196305 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:14.253366 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.253329 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:14.399304 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.399276 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d79d5b4d-54j5t"] Aug 04 20:44:14.403037 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:44:14.403001 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod255432c6_bf0d_44cb_b2f3_7983d6bba5b9.slice/crio-4428c8d4acef22336991244e15fd32bc54f8721e9b112066eff14c1ad90d115c WatchSource:0}: Error finding container 4428c8d4acef22336991244e15fd32bc54f8721e9b112066eff14c1ad90d115c: Status 404 returned error can't find the container with id 4428c8d4acef22336991244e15fd32bc54f8721e9b112066eff14c1ad90d115c Aug 04 20:44:14.736863 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.736733 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:14.800326 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:14.800283 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:15.255009 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:15.254971 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d79d5b4d-54j5t" event={"ID":"255432c6-bf0d-44cb-b2f3-7983d6bba5b9","Type":"ContainerStarted","Data":"8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa"} Aug 04 20:44:15.255453 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:15.255027 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d79d5b4d-54j5t" event={"ID":"255432c6-bf0d-44cb-b2f3-7983d6bba5b9","Type":"ContainerStarted","Data":"4428c8d4acef22336991244e15fd32bc54f8721e9b112066eff14c1ad90d115c"} Aug 04 20:44:15.275070 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:15.275024 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d79d5b4d-54j5t" podStartSLOduration=2.275007284 podStartE2EDuration="2.275007284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:44:15.273833191 +0000 UTC m=+189.420703757" watchObservedRunningTime="2026-08-04 20:44:15.275007284 +0000 UTC m=+189.421877840" Aug 04 20:44:24.254300 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:24.254248 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:24.254716 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:24.254327 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:24.259387 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:24.259364 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:24.290261 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:24.290229 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:44:24.339085 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:24.339044 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cd8bfd658-rsrkj"] Aug 04 20:44:24.456677 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:24.456637 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:24.555840 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:24.555712 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:24.630935 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:24.630881 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:44:49.360354 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.360293 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cd8bfd658-rsrkj" podUID="0a2950ef-3b3c-426d-a1cb-705dab37ec52" containerName="console" containerID="cri-o://decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9" gracePeriod=15 Aug 04 20:44:49.598307 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.598276 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:44:49.716890 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.716806 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-service-ca\") pod \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " Aug 04 20:44:49.716890 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.716846 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-trusted-ca-bundle\") pod \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " Aug 04 20:44:49.716890 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.716868 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-config\") pod \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " Aug 04 20:44:49.717158 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.716919 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5vhz\" (UniqueName: \"kubernetes.io/projected/0a2950ef-3b3c-426d-a1cb-705dab37ec52-kube-api-access-x5vhz\") pod \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " Aug 04 20:44:49.717158 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.716979 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-oauth-config\") pod \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " Aug 04 20:44:49.717158 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.716997 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-serving-cert\") pod \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " Aug 04 20:44:49.717158 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.717036 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-oauth-serving-cert\") pod \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\" (UID: \"0a2950ef-3b3c-426d-a1cb-705dab37ec52\") " Aug 04 20:44:49.717366 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.717254 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-service-ca" (OuterVolumeSpecName: "service-ca") pod "0a2950ef-3b3c-426d-a1cb-705dab37ec52" (UID: "0a2950ef-3b3c-426d-a1cb-705dab37ec52"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:44:49.717366 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.717293 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-config" (OuterVolumeSpecName: "console-config") pod "0a2950ef-3b3c-426d-a1cb-705dab37ec52" (UID: "0a2950ef-3b3c-426d-a1cb-705dab37ec52"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:44:49.717366 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.717304 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0a2950ef-3b3c-426d-a1cb-705dab37ec52" (UID: "0a2950ef-3b3c-426d-a1cb-705dab37ec52"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:44:49.717579 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.717561 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0a2950ef-3b3c-426d-a1cb-705dab37ec52" (UID: "0a2950ef-3b3c-426d-a1cb-705dab37ec52"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:44:49.719333 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.719304 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0a2950ef-3b3c-426d-a1cb-705dab37ec52" (UID: "0a2950ef-3b3c-426d-a1cb-705dab37ec52"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 20:44:49.719333 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.719325 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0a2950ef-3b3c-426d-a1cb-705dab37ec52" (UID: "0a2950ef-3b3c-426d-a1cb-705dab37ec52"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 20:44:49.719473 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.719322 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a2950ef-3b3c-426d-a1cb-705dab37ec52-kube-api-access-x5vhz" (OuterVolumeSpecName: "kube-api-access-x5vhz") pod "0a2950ef-3b3c-426d-a1cb-705dab37ec52" (UID: "0a2950ef-3b3c-426d-a1cb-705dab37ec52"). InnerVolumeSpecName "kube-api-access-x5vhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 20:44:49.818360 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.818329 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5vhz\" (UniqueName: \"kubernetes.io/projected/0a2950ef-3b3c-426d-a1cb-705dab37ec52-kube-api-access-x5vhz\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:44:49.818360 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.818354 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-oauth-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:44:49.818360 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.818364 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-serving-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:44:49.818561 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.818372 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-oauth-serving-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:44:49.818561 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.818380 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:44:49.818561 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.818389 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-trusted-ca-bundle\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:44:49.818561 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:49.818397 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a2950ef-3b3c-426d-a1cb-705dab37ec52-console-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:44:50.383653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.383616 2576 generic.go:356] "Generic (PLEG): container finished" podID="0a2950ef-3b3c-426d-a1cb-705dab37ec52" containerID="decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9" exitCode=0 Aug 04 20:44:50.383653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.383656 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd8bfd658-rsrkj" Aug 04 20:44:50.384145 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.383688 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd8bfd658-rsrkj" event={"ID":"0a2950ef-3b3c-426d-a1cb-705dab37ec52","Type":"ContainerDied","Data":"decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9"} Aug 04 20:44:50.384145 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.383734 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd8bfd658-rsrkj" event={"ID":"0a2950ef-3b3c-426d-a1cb-705dab37ec52","Type":"ContainerDied","Data":"28821ae2a3ee5cf2bf4c544a03830eb147089afc4c0c012648dbdb3a1451f8e7"} Aug 04 20:44:50.384145 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.383758 2576 scope.go:117] "RemoveContainer" containerID="decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9" Aug 04 20:44:50.394941 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.394882 2576 scope.go:117] "RemoveContainer" containerID="decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9" Aug 04 20:44:50.395292 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:44:50.395266 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9\": container with ID starting with decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9 not found: ID does not exist" containerID="decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9" Aug 04 20:44:50.395372 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.395303 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9"} err="failed to get container status \"decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9\": rpc error: code = NotFound desc = could not find container \"decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9\": container with ID starting with decd68cbd7b6d003ccfb113aef42e4c36cb0b8a88c4dbe5234354d7ef346a6c9 not found: ID does not exist" Aug 04 20:44:50.408975 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.408950 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cd8bfd658-rsrkj"] Aug 04 20:44:50.411234 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.411214 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cd8bfd658-rsrkj"] Aug 04 20:44:50.438847 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:44:50.438821 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a2950ef-3b3c-426d-a1cb-705dab37ec52" path="/var/lib/kubelet/pods/0a2950ef-3b3c-426d-a1cb-705dab37ec52/volumes" Aug 04 20:45:02.898114 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:02.898082 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:02.957353 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:02.957322 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:21.153432 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:21.153389 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:21.227773 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:21.227739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:22.633234 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.633202 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bc8cd488b-vtpmn"] Aug 04 20:45:22.633585 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.633497 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2950ef-3b3c-426d-a1cb-705dab37ec52" containerName="console" Aug 04 20:45:22.633585 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.633511 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2950ef-3b3c-426d-a1cb-705dab37ec52" containerName="console" Aug 04 20:45:22.633585 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.633556 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a2950ef-3b3c-426d-a1cb-705dab37ec52" containerName="console" Aug 04 20:45:22.637436 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.637418 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.651449 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.651424 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc8cd488b-vtpmn"] Aug 04 20:45:22.665402 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.665372 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-service-ca\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.665519 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.665406 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-console-config\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.665519 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.665433 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-oauth-serving-cert\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.665519 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.665500 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-serving-cert\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.665653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.665531 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-oauth-config\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.665653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.665578 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-trusted-ca-bundle\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.665653 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.665612 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zltbz\" (UniqueName: \"kubernetes.io/projected/191666c7-475e-4ddf-93cd-0106fe9f2134-kube-api-access-zltbz\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.706776 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.706745 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:22.766699 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.766667 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-service-ca\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.766812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.766708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-console-config\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.766812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.766728 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-oauth-serving-cert\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.766812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.766752 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-serving-cert\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.766985 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.766888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-oauth-config\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.767028 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.766983 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-trusted-ca-bundle\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.767028 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.767011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zltbz\" (UniqueName: \"kubernetes.io/projected/191666c7-475e-4ddf-93cd-0106fe9f2134-kube-api-access-zltbz\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.767540 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.767516 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-oauth-serving-cert\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.767618 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.767536 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-service-ca\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.767776 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.767757 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-console-config\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.768097 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.768071 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-trusted-ca-bundle\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.769375 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.769350 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-oauth-config\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.769476 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.769459 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-serving-cert\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.777240 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.777219 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:22.778889 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.778869 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zltbz\" (UniqueName: \"kubernetes.io/projected/191666c7-475e-4ddf-93cd-0106fe9f2134-kube-api-access-zltbz\") pod \"console-6bc8cd488b-vtpmn\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:22.946385 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:22.946296 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:23.076662 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:23.076636 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc8cd488b-vtpmn"] Aug 04 20:45:23.078812 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:45:23.078784 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191666c7_475e_4ddf_93cd_0106fe9f2134.slice/crio-46b56c46d7c360b9eb47631eabc173a0a3b2c045f2f203650d568d107ec6ca2e WatchSource:0}: Error finding container 46b56c46d7c360b9eb47631eabc173a0a3b2c045f2f203650d568d107ec6ca2e: Status 404 returned error can't find the container with id 46b56c46d7c360b9eb47631eabc173a0a3b2c045f2f203650d568d107ec6ca2e Aug 04 20:45:23.499068 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:23.499030 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc8cd488b-vtpmn" event={"ID":"191666c7-475e-4ddf-93cd-0106fe9f2134","Type":"ContainerStarted","Data":"acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782"} Aug 04 20:45:23.499268 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:23.499084 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc8cd488b-vtpmn" event={"ID":"191666c7-475e-4ddf-93cd-0106fe9f2134","Type":"ContainerStarted","Data":"46b56c46d7c360b9eb47631eabc173a0a3b2c045f2f203650d568d107ec6ca2e"} Aug 04 20:45:23.520130 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:23.520080 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bc8cd488b-vtpmn" podStartSLOduration=1.520063814 podStartE2EDuration="1.520063814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:45:23.518686541 +0000 UTC m=+257.665557096" watchObservedRunningTime="2026-08-04 20:45:23.520063814 +0000 UTC m=+257.666934368" Aug 04 20:45:32.946836 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:32.946731 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:32.946836 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:32.946798 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:32.951496 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:32.951470 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:33.158836 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:33.158798 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:33.340142 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:33.340076 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:33.539699 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:33.539674 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:45:33.591383 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:33.591290 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d79d5b4d-54j5t"] Aug 04 20:45:33.789403 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:33.789367 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:33.862410 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:33.862336 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:38.722352 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:38.722308 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:38.810859 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:38.810816 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:38.911241 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:38.911187 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:45:58.610304 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.610246 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d79d5b4d-54j5t" podUID="255432c6-bf0d-44cb-b2f3-7983d6bba5b9" containerName="console" containerID="cri-o://8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa" gracePeriod=15 Aug 04 20:45:58.844383 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.844361 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:45:58.954306 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954213 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-config\") pod \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " Aug 04 20:45:58.954306 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954278 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-oauth-config\") pod \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " Aug 04 20:45:58.954306 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954300 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-serving-cert\") pod \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " Aug 04 20:45:58.954526 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954460 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-trusted-ca-bundle\") pod \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " Aug 04 20:45:58.954591 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954527 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npcrr\" (UniqueName: \"kubernetes.io/projected/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-kube-api-access-npcrr\") pod \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " Aug 04 20:45:58.954591 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954563 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-service-ca\") pod \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " Aug 04 20:45:58.954692 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954598 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-oauth-serving-cert\") pod \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\" (UID: \"255432c6-bf0d-44cb-b2f3-7983d6bba5b9\") " Aug 04 20:45:58.954692 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954621 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-config" (OuterVolumeSpecName: "console-config") pod "255432c6-bf0d-44cb-b2f3-7983d6bba5b9" (UID: "255432c6-bf0d-44cb-b2f3-7983d6bba5b9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:45:58.954867 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954847 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:45:58.954965 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.954928 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "255432c6-bf0d-44cb-b2f3-7983d6bba5b9" (UID: "255432c6-bf0d-44cb-b2f3-7983d6bba5b9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:45:58.955216 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.955186 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-service-ca" (OuterVolumeSpecName: "service-ca") pod "255432c6-bf0d-44cb-b2f3-7983d6bba5b9" (UID: "255432c6-bf0d-44cb-b2f3-7983d6bba5b9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:45:58.955372 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.955352 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "255432c6-bf0d-44cb-b2f3-7983d6bba5b9" (UID: "255432c6-bf0d-44cb-b2f3-7983d6bba5b9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:45:58.956718 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.956697 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "255432c6-bf0d-44cb-b2f3-7983d6bba5b9" (UID: "255432c6-bf0d-44cb-b2f3-7983d6bba5b9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 20:45:58.956789 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.956751 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "255432c6-bf0d-44cb-b2f3-7983d6bba5b9" (UID: "255432c6-bf0d-44cb-b2f3-7983d6bba5b9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 20:45:58.956789 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:58.956765 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-kube-api-access-npcrr" (OuterVolumeSpecName: "kube-api-access-npcrr") pod "255432c6-bf0d-44cb-b2f3-7983d6bba5b9" (UID: "255432c6-bf0d-44cb-b2f3-7983d6bba5b9"). InnerVolumeSpecName "kube-api-access-npcrr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 20:45:59.055257 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.055225 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-oauth-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:45:59.055257 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.055252 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-console-serving-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:45:59.055257 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.055262 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-trusted-ca-bundle\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:45:59.055472 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.055271 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npcrr\" (UniqueName: \"kubernetes.io/projected/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-kube-api-access-npcrr\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:45:59.055472 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.055282 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:45:59.055472 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.055290 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/255432c6-bf0d-44cb-b2f3-7983d6bba5b9-oauth-serving-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:45:59.636882 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.636846 2576 generic.go:356] "Generic (PLEG): container finished" podID="255432c6-bf0d-44cb-b2f3-7983d6bba5b9" containerID="8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa" exitCode=0 Aug 04 20:45:59.637350 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.636929 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d79d5b4d-54j5t" Aug 04 20:45:59.637350 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.636939 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d79d5b4d-54j5t" event={"ID":"255432c6-bf0d-44cb-b2f3-7983d6bba5b9","Type":"ContainerDied","Data":"8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa"} Aug 04 20:45:59.637350 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.637015 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d79d5b4d-54j5t" event={"ID":"255432c6-bf0d-44cb-b2f3-7983d6bba5b9","Type":"ContainerDied","Data":"4428c8d4acef22336991244e15fd32bc54f8721e9b112066eff14c1ad90d115c"} Aug 04 20:45:59.637350 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.637037 2576 scope.go:117] "RemoveContainer" containerID="8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa" Aug 04 20:45:59.646008 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.645991 2576 scope.go:117] "RemoveContainer" containerID="8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa" Aug 04 20:45:59.646247 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:45:59.646221 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa\": container with ID starting with 8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa not found: ID does not exist" containerID="8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa" Aug 04 20:45:59.646310 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.646259 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa"} err="failed to get container status \"8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa\": rpc error: code = NotFound desc = could not find container \"8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa\": container with ID starting with 8a7be4e89e5716659514c79012d74468496167fc97d0688236ec92833caf99aa not found: ID does not exist" Aug 04 20:45:59.659176 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.659151 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d79d5b4d-54j5t"] Aug 04 20:45:59.665373 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:45:59.665346 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d79d5b4d-54j5t"] Aug 04 20:46:00.438580 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:46:00.438547 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="255432c6-bf0d-44cb-b2f3-7983d6bba5b9" path="/var/lib/kubelet/pods/255432c6-bf0d-44cb-b2f3-7983d6bba5b9/volumes" Aug 04 20:46:05.341149 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:46:05.341114 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:46:06.265780 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:46:06.265745 2576 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 20:46:36.276199 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:46:36.276164 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:46:38.497887 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:46:38.497853 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:46:38.591198 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:46:38.591142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:05.487263 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:05.487176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:06.500544 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:06.500512 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:09.100701 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:09.100643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:09.201618 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:09.201584 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:10.300804 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:10.300768 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:10.400757 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:10.400729 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:10.909443 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:10.909412 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:11.011116 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:11.011081 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:12.291268 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.291230 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5rw8d"] Aug 04 20:47:12.291649 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.291505 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="255432c6-bf0d-44cb-b2f3-7983d6bba5b9" containerName="console" Aug 04 20:47:12.291649 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.291515 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="255432c6-bf0d-44cb-b2f3-7983d6bba5b9" containerName="console" Aug 04 20:47:12.291649 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.291572 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="255432c6-bf0d-44cb-b2f3-7983d6bba5b9" containerName="console" Aug 04 20:47:12.294544 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.294527 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:12.297305 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.297279 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Aug 04 20:47:12.298094 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.298074 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Aug 04 20:47:12.298195 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.298073 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sm54c\"" Aug 04 20:47:12.304464 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.304442 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5rw8d"] Aug 04 20:47:12.341734 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.341696 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjplr\" (UniqueName: \"kubernetes.io/projected/a58a92d2-5fca-4a8a-a9d9-d47682f29591-kube-api-access-rjplr\") pod \"cert-manager-webhook-6486ffd997-5rw8d\" (UID: \"a58a92d2-5fca-4a8a-a9d9-d47682f29591\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:12.341874 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.341746 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a58a92d2-5fca-4a8a-a9d9-d47682f29591-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5rw8d\" (UID: \"a58a92d2-5fca-4a8a-a9d9-d47682f29591\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:12.442685 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.442652 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjplr\" (UniqueName: \"kubernetes.io/projected/a58a92d2-5fca-4a8a-a9d9-d47682f29591-kube-api-access-rjplr\") pod \"cert-manager-webhook-6486ffd997-5rw8d\" (UID: \"a58a92d2-5fca-4a8a-a9d9-d47682f29591\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:12.442844 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.442698 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a58a92d2-5fca-4a8a-a9d9-d47682f29591-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5rw8d\" (UID: \"a58a92d2-5fca-4a8a-a9d9-d47682f29591\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:12.451471 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.451448 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a58a92d2-5fca-4a8a-a9d9-d47682f29591-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5rw8d\" (UID: \"a58a92d2-5fca-4a8a-a9d9-d47682f29591\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:12.452020 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.452003 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjplr\" (UniqueName: \"kubernetes.io/projected/a58a92d2-5fca-4a8a-a9d9-d47682f29591-kube-api-access-rjplr\") pod \"cert-manager-webhook-6486ffd997-5rw8d\" (UID: \"a58a92d2-5fca-4a8a-a9d9-d47682f29591\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:12.522531 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.522491 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:12.603535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.603508 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:12.718137 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.718105 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:12.735583 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.735526 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5rw8d"] Aug 04 20:47:12.739399 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:47:12.739372 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda58a92d2_5fca_4a8a_a9d9_d47682f29591.slice/crio-14713dda82c3ab55755b883cfaaeabb9c78c375f43639044c217daee028de286 WatchSource:0}: Error finding container 14713dda82c3ab55755b883cfaaeabb9c78c375f43639044c217daee028de286: Status 404 returned error can't find the container with id 14713dda82c3ab55755b883cfaaeabb9c78c375f43639044c217daee028de286 Aug 04 20:47:12.741399 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.741373 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 20:47:12.898247 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:12.898163 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" event={"ID":"a58a92d2-5fca-4a8a-a9d9-d47682f29591","Type":"ContainerStarted","Data":"14713dda82c3ab55755b883cfaaeabb9c78c375f43639044c217daee028de286"} Aug 04 20:47:15.892512 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:15.892478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:15.994728 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:15.994694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:20.493055 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:20.492978 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:20.596247 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:20.596211 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:20.719059 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:20.719031 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:20.932337 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:20.932301 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" event={"ID":"a58a92d2-5fca-4a8a-a9d9-d47682f29591","Type":"ContainerStarted","Data":"7e74a46318f9c31b7b565e960ab3979e1f2260d6fade8f2e36f6226be160dead"} Aug 04 20:47:20.932535 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:20.932423 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:20.949512 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:20.949458 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" podStartSLOduration=1.454997381 podStartE2EDuration="8.949443849s" totalImagesPullingTime="7.494446468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:47:20.949182255 +0000 UTC m=+375.096052811" watchObservedRunningTime="2026-08-04 20:47:20.949443849 +0000 UTC m=+375.096314404" Aug 04 20:47:22.308574 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:22.308487 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:22.409625 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:22.409528 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:26.941582 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:26.941553 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5rw8d" Aug 04 20:47:28.331458 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:28.331388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:28.434681 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:28.434644 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:30.446514 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:30.446484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:36.382053 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.382014 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f99ff488f-xsz8h"] Aug 04 20:47:36.385233 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.385210 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.395260 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.395238 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f99ff488f-xsz8h"] Aug 04 20:47:36.425632 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.425581 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3cc2fd20-ef93-472e-89a0-d56579654878-console-oauth-config\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.425807 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.425649 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3cc2fd20-ef93-472e-89a0-d56579654878-console-serving-cert\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.425807 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.425685 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjdkl\" (UniqueName: \"kubernetes.io/projected/3cc2fd20-ef93-472e-89a0-d56579654878-kube-api-access-hjdkl\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.425807 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.425725 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-console-config\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.425983 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.425811 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-trusted-ca-bundle\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.425983 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.425847 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-service-ca\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.425983 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.425871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-oauth-serving-cert\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.433371 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.433343 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:36.502269 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.502239 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:36.526743 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.526716 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-trusted-ca-bundle\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.526952 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.526750 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-service-ca\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.526952 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.526769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-oauth-serving-cert\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.526952 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.526800 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3cc2fd20-ef93-472e-89a0-d56579654878-console-oauth-config\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.526952 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.526828 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3cc2fd20-ef93-472e-89a0-d56579654878-console-serving-cert\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.526952 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.526855 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjdkl\" (UniqueName: \"kubernetes.io/projected/3cc2fd20-ef93-472e-89a0-d56579654878-kube-api-access-hjdkl\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.526952 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.526888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-console-config\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.527620 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.527594 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-service-ca\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.527766 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.527695 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-console-config\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.527841 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.527699 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-oauth-serving-cert\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.527841 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.527708 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cc2fd20-ef93-472e-89a0-d56579654878-trusted-ca-bundle\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.529415 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.529395 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3cc2fd20-ef93-472e-89a0-d56579654878-console-serving-cert\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.529474 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.529441 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3cc2fd20-ef93-472e-89a0-d56579654878-console-oauth-config\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.535192 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.535168 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjdkl\" (UniqueName: \"kubernetes.io/projected/3cc2fd20-ef93-472e-89a0-d56579654878-kube-api-access-hjdkl\") pod \"console-6f99ff488f-xsz8h\" (UID: \"3cc2fd20-ef93-472e-89a0-d56579654878\") " pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.693815 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.693708 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:36.821975 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.821945 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f99ff488f-xsz8h"] Aug 04 20:47:36.825223 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:47:36.825193 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc2fd20_ef93_472e_89a0_d56579654878.slice/crio-58f98c7887c82a89c0bb0670cb76b079ca358a918a0fbe483b596eeab641f9b3 WatchSource:0}: Error finding container 58f98c7887c82a89c0bb0670cb76b079ca358a918a0fbe483b596eeab641f9b3: Status 404 returned error can't find the container with id 58f98c7887c82a89c0bb0670cb76b079ca358a918a0fbe483b596eeab641f9b3 Aug 04 20:47:36.995743 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.995652 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f99ff488f-xsz8h" event={"ID":"3cc2fd20-ef93-472e-89a0-d56579654878","Type":"ContainerStarted","Data":"d349cc1a90cb5b73014e3e745db13a680e195b82a534d34acdaa171bd2e1ac77"} Aug 04 20:47:36.995743 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:36.995710 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f99ff488f-xsz8h" event={"ID":"3cc2fd20-ef93-472e-89a0-d56579654878","Type":"ContainerStarted","Data":"58f98c7887c82a89c0bb0670cb76b079ca358a918a0fbe483b596eeab641f9b3"} Aug 04 20:47:37.018664 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:37.018600 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f99ff488f-xsz8h" podStartSLOduration=1.018582795 podStartE2EDuration="1.018582795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:47:37.017320869 +0000 UTC m=+391.164191423" watchObservedRunningTime="2026-08-04 20:47:37.018582795 +0000 UTC m=+391.165453352" Aug 04 20:47:46.300105 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.300069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:46.410272 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.410233 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:46.694361 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.694258 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:46.694361 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.694353 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:46.699392 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.699366 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:46.733463 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.733435 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:46.809812 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.809784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:46.821659 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.821632 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw"] Aug 04 20:47:46.824944 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.824929 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:46.827931 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.827893 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 04 20:47:46.828143 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.828127 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 04 20:47:46.828234 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.828168 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2p2tb\"" Aug 04 20:47:46.828603 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.828585 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 04 20:47:46.828688 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.828599 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 04 20:47:46.828688 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.828639 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 04 20:47:46.843273 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.843255 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw"] Aug 04 20:47:46.913109 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.913085 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a884dc58-5aa1-4881-9f01-a41bd5aea854-manager-config\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:46.913239 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.913115 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgg2s\" (UniqueName: \"kubernetes.io/projected/a884dc58-5aa1-4881-9f01-a41bd5aea854-kube-api-access-bgg2s\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:46.913239 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.913137 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a884dc58-5aa1-4881-9f01-a41bd5aea854-cert\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:46.913239 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.913193 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a884dc58-5aa1-4881-9f01-a41bd5aea854-metrics-cert\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:46.946033 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:46.945962 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:47.014291 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.014260 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a884dc58-5aa1-4881-9f01-a41bd5aea854-metrics-cert\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:47.014420 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.014337 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a884dc58-5aa1-4881-9f01-a41bd5aea854-manager-config\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:47.014420 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.014366 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgg2s\" (UniqueName: \"kubernetes.io/projected/a884dc58-5aa1-4881-9f01-a41bd5aea854-kube-api-access-bgg2s\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:47.014420 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.014396 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a884dc58-5aa1-4881-9f01-a41bd5aea854-cert\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:47.015113 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.015042 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a884dc58-5aa1-4881-9f01-a41bd5aea854-manager-config\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:47.016881 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.016851 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a884dc58-5aa1-4881-9f01-a41bd5aea854-cert\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:47.016997 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.016922 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a884dc58-5aa1-4881-9f01-a41bd5aea854-metrics-cert\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:47.027403 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.027384 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgg2s\" (UniqueName: \"kubernetes.io/projected/a884dc58-5aa1-4881-9f01-a41bd5aea854-kube-api-access-bgg2s\") pod \"lws-controller-manager-7564c45d7d-zzntw\" (UID: \"a884dc58-5aa1-4881-9f01-a41bd5aea854\") " pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:47.036478 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.036460 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f99ff488f-xsz8h" Aug 04 20:47:47.134304 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.134269 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:47.148643 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.148619 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bc8cd488b-vtpmn"] Aug 04 20:47:47.308899 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.308870 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw"] Aug 04 20:47:47.312116 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:47:47.312091 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda884dc58_5aa1_4881_9f01_a41bd5aea854.slice/crio-68cdf3970428ab84b5c3dbc0fa76e3040bbeceb0d8a6c7040bd55b3f7770f28b WatchSource:0}: Error finding container 68cdf3970428ab84b5c3dbc0fa76e3040bbeceb0d8a6c7040bd55b3f7770f28b: Status 404 returned error can't find the container with id 68cdf3970428ab84b5c3dbc0fa76e3040bbeceb0d8a6c7040bd55b3f7770f28b Aug 04 20:47:47.391194 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.391167 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:47.475252 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.475175 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:47.617192 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.617162 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2"] Aug 04 20:47:47.621631 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.621609 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.624727 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.624708 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 20:47:47.625187 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.625168 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 20:47:47.625332 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.625231 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 20:47:47.625472 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.625455 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4mstc\"" Aug 04 20:47:47.625551 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.625468 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 20:47:47.637083 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.637062 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2"] Aug 04 20:47:47.721493 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.721464 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/815d611d-3da2-4318-b23d-4816c7633ae1-tmp\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.721635 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.721504 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/815d611d-3da2-4318-b23d-4816c7633ae1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.721635 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.721571 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/815d611d-3da2-4318-b23d-4816c7633ae1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.721635 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.721589 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwdl2\" (UniqueName: \"kubernetes.io/projected/815d611d-3da2-4318-b23d-4816c7633ae1-kube-api-access-bwdl2\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.721635 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.721616 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/815d611d-3da2-4318-b23d-4816c7633ae1-manifests\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.747697 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.747630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:47.822120 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.822093 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwdl2\" (UniqueName: \"kubernetes.io/projected/815d611d-3da2-4318-b23d-4816c7633ae1-kube-api-access-bwdl2\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.822279 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.822140 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/815d611d-3da2-4318-b23d-4816c7633ae1-manifests\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.822279 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.822164 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/815d611d-3da2-4318-b23d-4816c7633ae1-tmp\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.822279 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.822193 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/815d611d-3da2-4318-b23d-4816c7633ae1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.822279 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.822238 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/815d611d-3da2-4318-b23d-4816c7633ae1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.822524 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.822488 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/815d611d-3da2-4318-b23d-4816c7633ae1-manifests\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.824678 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.824650 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/815d611d-3da2-4318-b23d-4816c7633ae1-tmp\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.824789 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.824770 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/815d611d-3da2-4318-b23d-4816c7633ae1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.824886 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.824871 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/815d611d-3da2-4318-b23d-4816c7633ae1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.833207 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.833190 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwdl2\" (UniqueName: \"kubernetes.io/projected/815d611d-3da2-4318-b23d-4816c7633ae1-kube-api-access-bwdl2\") pod \"opendatahub-operator-controller-manager-6889885b4f-zw2b2\" (UID: \"815d611d-3da2-4318-b23d-4816c7633ae1\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.862807 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.862785 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:47.932276 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.932252 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:47.947607 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:47.947583 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:48.038468 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:48.038436 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" event={"ID":"a884dc58-5aa1-4881-9f01-a41bd5aea854","Type":"ContainerStarted","Data":"68cdf3970428ab84b5c3dbc0fa76e3040bbeceb0d8a6c7040bd55b3f7770f28b"} Aug 04 20:47:48.075005 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:48.074973 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:48.099545 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:48.099523 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2"] Aug 04 20:47:48.101221 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:47:48.101191 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod815d611d_3da2_4318_b23d_4816c7633ae1.slice/crio-5ea3a3748af44975df8795f7a400724baadf88f0dea991b07b5d26d80073379b WatchSource:0}: Error finding container 5ea3a3748af44975df8795f7a400724baadf88f0dea991b07b5d26d80073379b: Status 404 returned error can't find the container with id 5ea3a3748af44975df8795f7a400724baadf88f0dea991b07b5d26d80073379b Aug 04 20:47:48.153446 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:48.153410 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:48.252672 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:48.252642 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:48.354829 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:48.354790 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:47:49.046225 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:49.046189 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" event={"ID":"815d611d-3da2-4318-b23d-4816c7633ae1","Type":"ContainerStarted","Data":"5ea3a3748af44975df8795f7a400724baadf88f0dea991b07b5d26d80073379b"} Aug 04 20:47:52.062244 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:52.062158 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" event={"ID":"815d611d-3da2-4318-b23d-4816c7633ae1","Type":"ContainerStarted","Data":"e834d92f7f5ffdc1315416dbb7a6994b6f3ae27214394c3af76e900e40ddb533"} Aug 04 20:47:52.063635 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:52.063610 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" event={"ID":"a884dc58-5aa1-4881-9f01-a41bd5aea854","Type":"ContainerStarted","Data":"13a9187b3007d1b7d46d7cb5242d2ae498706b202e2457dbc40f0a874efbba0b"} Aug 04 20:47:52.063761 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:52.063748 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:47:52.101971 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:52.101920 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" podStartSLOduration=1.7723157550000002 podStartE2EDuration="6.10188662s" totalImagesPullingTime="4.329570865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:47:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:47:52.100174403 +0000 UTC m=+406.247044985" watchObservedRunningTime="2026-08-04 20:47:52.10188662 +0000 UTC m=+406.248757174" Aug 04 20:47:53.069114 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:53.069084 2576 generic.go:356] "Generic (PLEG): container finished" podID="815d611d-3da2-4318-b23d-4816c7633ae1" containerID="e834d92f7f5ffdc1315416dbb7a6994b6f3ae27214394c3af76e900e40ddb533" exitCode=0 Aug 04 20:47:53.069474 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:53.069168 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" event={"ID":"815d611d-3da2-4318-b23d-4816c7633ae1","Type":"ContainerDied","Data":"e834d92f7f5ffdc1315416dbb7a6994b6f3ae27214394c3af76e900e40ddb533"} Aug 04 20:47:54.074851 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:54.074815 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" event={"ID":"815d611d-3da2-4318-b23d-4816c7633ae1","Type":"ContainerStarted","Data":"3c8bbf547471819a3bf8a6dfb5bf365b788214ed4935ee0c408e1584c42c1f80"} Aug 04 20:47:54.075324 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:54.075058 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:47:54.100449 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:47:54.100400 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" podStartSLOduration=2.054051833 podStartE2EDuration="7.100385044s" totalImagesPullingTime="3.973815127s" totalInitContainerRuntime="1.072518084s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:47:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:47:54.09923907 +0000 UTC m=+408.246109625" watchObservedRunningTime="2026-08-04 20:47:54.100385044 +0000 UTC m=+408.247255644" Aug 04 20:48:03.071429 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:03.071397 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7564c45d7d-zzntw" Aug 04 20:48:05.080880 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:05.080850 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-zw2b2" Aug 04 20:48:07.400141 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:07.400102 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:07.526056 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:07.526018 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:07.877425 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:07.877391 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:08.150165 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:08.150086 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:08.228219 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:08.228187 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:08.543430 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:08.543347 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:08.810683 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:08.810593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:09.222540 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:09.222511 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:09.481509 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:09.481425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:09.551295 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:09.551259 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:10.627377 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:10.627346 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:12.171676 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.171637 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bc8cd488b-vtpmn" podUID="191666c7-475e-4ddf-93cd-0106fe9f2134" containerName="console" containerID="cri-o://acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782" gracePeriod=15 Aug 04 20:48:12.407837 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.407811 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:48:12.534083 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.533995 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-console-config\") pod \"191666c7-475e-4ddf-93cd-0106fe9f2134\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " Aug 04 20:48:12.534083 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534033 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zltbz\" (UniqueName: \"kubernetes.io/projected/191666c7-475e-4ddf-93cd-0106fe9f2134-kube-api-access-zltbz\") pod \"191666c7-475e-4ddf-93cd-0106fe9f2134\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " Aug 04 20:48:12.534083 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534086 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-trusted-ca-bundle\") pod \"191666c7-475e-4ddf-93cd-0106fe9f2134\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " Aug 04 20:48:12.534337 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534113 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-service-ca\") pod \"191666c7-475e-4ddf-93cd-0106fe9f2134\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " Aug 04 20:48:12.534337 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534135 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-oauth-serving-cert\") pod \"191666c7-475e-4ddf-93cd-0106fe9f2134\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " Aug 04 20:48:12.534337 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534154 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-serving-cert\") pod \"191666c7-475e-4ddf-93cd-0106fe9f2134\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " Aug 04 20:48:12.534337 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534189 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-oauth-config\") pod \"191666c7-475e-4ddf-93cd-0106fe9f2134\" (UID: \"191666c7-475e-4ddf-93cd-0106fe9f2134\") " Aug 04 20:48:12.534482 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534440 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-console-config" (OuterVolumeSpecName: "console-config") pod "191666c7-475e-4ddf-93cd-0106fe9f2134" (UID: "191666c7-475e-4ddf-93cd-0106fe9f2134"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:48:12.534553 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534530 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "191666c7-475e-4ddf-93cd-0106fe9f2134" (UID: "191666c7-475e-4ddf-93cd-0106fe9f2134"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:48:12.534801 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534777 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-service-ca" (OuterVolumeSpecName: "service-ca") pod "191666c7-475e-4ddf-93cd-0106fe9f2134" (UID: "191666c7-475e-4ddf-93cd-0106fe9f2134"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:48:12.534801 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.534775 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "191666c7-475e-4ddf-93cd-0106fe9f2134" (UID: "191666c7-475e-4ddf-93cd-0106fe9f2134"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 20:48:12.536529 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.536507 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "191666c7-475e-4ddf-93cd-0106fe9f2134" (UID: "191666c7-475e-4ddf-93cd-0106fe9f2134"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 20:48:12.536606 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.536575 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "191666c7-475e-4ddf-93cd-0106fe9f2134" (UID: "191666c7-475e-4ddf-93cd-0106fe9f2134"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 20:48:12.536667 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.536639 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/191666c7-475e-4ddf-93cd-0106fe9f2134-kube-api-access-zltbz" (OuterVolumeSpecName: "kube-api-access-zltbz") pod "191666c7-475e-4ddf-93cd-0106fe9f2134" (UID: "191666c7-475e-4ddf-93cd-0106fe9f2134"). InnerVolumeSpecName "kube-api-access-zltbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 20:48:12.635601 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.635556 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-trusted-ca-bundle\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:48:12.635601 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.635595 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:48:12.635601 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.635604 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-oauth-serving-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:48:12.635601 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.635613 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-serving-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:48:12.635891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.635621 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/191666c7-475e-4ddf-93cd-0106fe9f2134-console-oauth-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:48:12.635891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.635629 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/191666c7-475e-4ddf-93cd-0106fe9f2134-console-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:48:12.635891 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:12.635638 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zltbz\" (UniqueName: \"kubernetes.io/projected/191666c7-475e-4ddf-93cd-0106fe9f2134-kube-api-access-zltbz\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 20:48:13.154071 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:13.154027 2576 generic.go:356] "Generic (PLEG): container finished" podID="191666c7-475e-4ddf-93cd-0106fe9f2134" containerID="acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782" exitCode=0 Aug 04 20:48:13.154256 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:13.154088 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc8cd488b-vtpmn" event={"ID":"191666c7-475e-4ddf-93cd-0106fe9f2134","Type":"ContainerDied","Data":"acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782"} Aug 04 20:48:13.154256 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:13.154096 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc8cd488b-vtpmn" Aug 04 20:48:13.154256 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:13.154169 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc8cd488b-vtpmn" event={"ID":"191666c7-475e-4ddf-93cd-0106fe9f2134","Type":"ContainerDied","Data":"46b56c46d7c360b9eb47631eabc173a0a3b2c045f2f203650d568d107ec6ca2e"} Aug 04 20:48:13.154372 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:13.154253 2576 scope.go:117] "RemoveContainer" containerID="acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782" Aug 04 20:48:13.162862 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:13.162844 2576 scope.go:117] "RemoveContainer" containerID="acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782" Aug 04 20:48:13.163131 ip-10-0-139-54 kubenswrapper[2576]: E0804 20:48:13.163113 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782\": container with ID starting with acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782 not found: ID does not exist" containerID="acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782" Aug 04 20:48:13.163182 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:13.163139 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782"} err="failed to get container status \"acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782\": rpc error: code = NotFound desc = could not find container \"acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782\": container with ID starting with acc218c1cbcfd06234c6d4d9ad294ce6d95c5b5a938241f82eb7e9680ce40782 not found: ID does not exist" Aug 04 20:48:13.175668 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:13.175642 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bc8cd488b-vtpmn"] Aug 04 20:48:13.179706 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:13.179680 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bc8cd488b-vtpmn"] Aug 04 20:48:14.439180 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:14.439145 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="191666c7-475e-4ddf-93cd-0106fe9f2134" path="/var/lib/kubelet/pods/191666c7-475e-4ddf-93cd-0106fe9f2134/volumes" Aug 04 20:48:15.478817 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.478787 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:15.579215 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.579186 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:15.737186 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.737112 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc"] Aug 04 20:48:15.737421 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.737408 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="191666c7-475e-4ddf-93cd-0106fe9f2134" containerName="console" Aug 04 20:48:15.737471 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.737422 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="191666c7-475e-4ddf-93cd-0106fe9f2134" containerName="console" Aug 04 20:48:15.737508 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.737476 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="191666c7-475e-4ddf-93cd-0106fe9f2134" containerName="console" Aug 04 20:48:15.741618 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.741600 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.745096 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.745074 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 20:48:15.745222 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.745125 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 20:48:15.745222 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.745181 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-49w79\"" Aug 04 20:48:15.745955 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.745939 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 20:48:15.761986 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.761966 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc"] Aug 04 20:48:15.780360 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.780328 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:15.865139 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.865110 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.865265 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.865146 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.865265 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.865165 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.865265 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.865185 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2d6c\" (UniqueName: \"kubernetes.io/projected/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-kube-api-access-d2d6c\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.865265 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.865257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.865464 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.865280 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.865464 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.865336 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.865464 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.865398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.865464 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.865436 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.918604 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.918572 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:15.966053 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966025 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966182 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966059 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966182 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966087 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966182 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966117 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966182 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966149 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966182 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966172 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966192 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2d6c\" (UniqueName: \"kubernetes.io/projected/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-kube-api-access-d2d6c\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966224 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966248 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966475 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966508 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966732 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.966929 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966802 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.967167 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.966961 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.968685 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.968664 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.969220 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.969205 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.976528 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.976486 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:15.976641 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:15.976543 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2d6c\" (UniqueName: \"kubernetes.io/projected/4aa57cb2-e32d-4c49-a86f-f8dbac7273ce-kube-api-access-d2d6c\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc\" (UID: \"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:16.019639 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.019562 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:16.051494 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.051449 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:16.189774 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.189750 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc"] Aug 04 20:48:16.192133 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:48:16.192104 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa57cb2_e32d_4c49_a86f_f8dbac7273ce.slice/crio-e6e0e5cdf0075b14e53d294f6bb0680d1da93a58c6c7861a115c4710e214c43e WatchSource:0}: Error finding container e6e0e5cdf0075b14e53d294f6bb0680d1da93a58c6c7861a115c4710e214c43e: Status 404 returned error can't find the container with id e6e0e5cdf0075b14e53d294f6bb0680d1da93a58c6c7861a115c4710e214c43e Aug 04 20:48:16.587864 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.587822 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86"] Aug 04 20:48:16.592272 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.592245 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.595094 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.595072 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Aug 04 20:48:16.595226 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.595116 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c82v6\"" Aug 04 20:48:16.595468 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.595453 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Aug 04 20:48:16.595536 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.595478 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Aug 04 20:48:16.612263 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.612237 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86"] Aug 04 20:48:16.773301 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.773268 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.773457 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.773307 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3160f39-c643-4471-a95f-bbf6b1c71c0c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.773457 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.773358 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wktxm\" (UniqueName: \"kubernetes.io/projected/b3160f39-c643-4471-a95f-bbf6b1c71c0c-kube-api-access-wktxm\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.773457 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.773399 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.773457 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.773419 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b3160f39-c643-4471-a95f-bbf6b1c71c0c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.773592 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.773480 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.773592 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.773509 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.874140 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.874103 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.874307 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.874145 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b3160f39-c643-4471-a95f-bbf6b1c71c0c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.874307 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.874194 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.874307 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.874225 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.874307 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.874262 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.874307 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.874295 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3160f39-c643-4471-a95f-bbf6b1c71c0c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.874628 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.874336 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wktxm\" (UniqueName: \"kubernetes.io/projected/b3160f39-c643-4471-a95f-bbf6b1c71c0c-kube-api-access-wktxm\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.875055 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.874939 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.876883 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.876863 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.876991 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.876865 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b3160f39-c643-4471-a95f-bbf6b1c71c0c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.877196 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.877175 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3160f39-c643-4471-a95f-bbf6b1c71c0c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.877308 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.877293 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.886328 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.886307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3160f39-c643-4471-a95f-bbf6b1c71c0c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.889377 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.889356 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wktxm\" (UniqueName: \"kubernetes.io/projected/b3160f39-c643-4471-a95f-bbf6b1c71c0c-kube-api-access-wktxm\") pod \"istiod-openshift-gateway-5677ccb49b-rdm86\" (UID: \"b3160f39-c643-4471-a95f-bbf6b1c71c0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:16.902245 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:16.902226 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:17.056623 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:17.056241 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86"] Aug 04 20:48:17.059104 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:48:17.059079 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3160f39_c643_4471_a95f_bbf6b1c71c0c.slice/crio-2f38c2cf54ec03eedf42425ffd599f6c28dfb561ac963cf52f695007d3884630 WatchSource:0}: Error finding container 2f38c2cf54ec03eedf42425ffd599f6c28dfb561ac963cf52f695007d3884630: Status 404 returned error can't find the container with id 2f38c2cf54ec03eedf42425ffd599f6c28dfb561ac963cf52f695007d3884630 Aug 04 20:48:17.172899 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:17.172821 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" event={"ID":"b3160f39-c643-4471-a95f-bbf6b1c71c0c","Type":"ContainerStarted","Data":"2f38c2cf54ec03eedf42425ffd599f6c28dfb561ac963cf52f695007d3884630"} Aug 04 20:48:17.173993 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:17.173970 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" event={"ID":"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce","Type":"ContainerStarted","Data":"e6e0e5cdf0075b14e53d294f6bb0680d1da93a58c6c7861a115c4710e214c43e"} Aug 04 20:48:19.646139 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:19.646101 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:19.765879 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:19.765830 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:20.109069 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:20.109040 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:20.212416 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:20.212362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:20.686669 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:20.686619 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:48:20.687081 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:20.686691 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:48:20.690209 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:20.690185 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:48:20.690289 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:20.690242 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:48:20.690289 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:20.690270 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:48:20.982640 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:20.982563 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:21.096684 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:21.096650 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:21.194275 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:21.194240 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:21.198508 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:21.198476 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" event={"ID":"b3160f39-c643-4471-a95f-bbf6b1c71c0c","Type":"ContainerStarted","Data":"7f883bafef66c6c6a8af6d93a6cc4506e4a875d1513005216d97aab5e23f1851"} Aug 04 20:48:21.198871 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:21.198844 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:21.201182 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:21.201116 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-rdm86 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 20:48:21.201182 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:21.201168 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" event={"ID":"4aa57cb2-e32d-4c49-a86f-f8dbac7273ce","Type":"ContainerStarted","Data":"80d28e1145a65687807d55f79e9813587c29a84f92541df621d0ffcdc27a8a04"} Aug 04 20:48:21.201182 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:21.201174 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" podUID="b3160f39-c643-4471-a95f-bbf6b1c71c0c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 20:48:21.244728 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:21.244624 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" podStartSLOduration=1.619554071 podStartE2EDuration="5.244605111s" totalImagesPullingTime="3.62505104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:48:21.24205593 +0000 UTC m=+435.388926497" watchObservedRunningTime="2026-08-04 20:48:21.244605111 +0000 UTC m=+435.391475671" Aug 04 20:48:21.283620 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:21.283566 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" podStartSLOduration=1.788085938 podStartE2EDuration="6.283547612s" totalImagesPullingTime="4.495461674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:48:21.282255603 +0000 UTC m=+435.429126162" watchObservedRunningTime="2026-08-04 20:48:21.283547612 +0000 UTC m=+435.430418168" Aug 04 20:48:22.052001 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:22.051963 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:22.056355 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:22.056330 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:22.207154 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:22.207118 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:22.207693 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:22.207668 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rdm86" Aug 04 20:48:22.208072 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:22.208056 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc" Aug 04 20:48:23.425826 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:23.425792 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:25.664271 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:25.664230 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:42.751490 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:42.751459 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:42.823715 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:42.823681 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:48:49.047005 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:48:49.046972 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:09.083725 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:09.083690 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:09.185325 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:09.185290 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:22.753201 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:22.753169 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:25.583854 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:25.583822 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:25.956509 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:25.956477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:26.355953 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:26.355895 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:27.202735 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:27.202700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:29.010447 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:29.010415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:29.105094 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:29.105055 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:29.966269 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:29.966236 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:31.611352 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:31.611320 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:31.741444 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:31.741413 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:32.358588 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:32.358555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:33.829094 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:33.829059 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:33.929246 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:33.929214 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:35.763257 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:35.763221 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:35.860432 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:35.860400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:37.792573 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:37.792523 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:38.135740 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:38.135707 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:40.364325 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:40.364284 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:40.465378 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:40.465347 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:40.582861 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:40.582824 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:40.660181 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:40.660106 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:41.554309 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:41.554279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:41.620881 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:41.620853 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:47.441204 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:47.441168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:47.541492 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:47.541459 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:52.185794 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:52.185757 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:52.301382 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:52.301349 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:52.414998 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:52.414962 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:54.778076 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:54.778044 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:49:54.882549 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:49:54.882508 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:00.233921 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:00.233814 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:00.334691 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:00.334659 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:00.595965 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:00.595936 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:00.688885 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:00.688851 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:01.017830 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:01.017735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:01.128205 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:01.128175 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:07.194060 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:07.194024 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:07.294801 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:07.294768 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:11.995294 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:11.995257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:12.096734 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:12.096704 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:16.734919 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.734879 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x"] Aug 04 20:50:16.738318 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.738295 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.740712 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.740692 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7mhwk\"" Aug 04 20:50:16.754954 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.754925 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x"] Aug 04 20:50:16.762655 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.762634 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:16.825630 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.825594 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.825630 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.825631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2lfb\" (UniqueName: \"kubernetes.io/projected/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-kube-api-access-c2lfb\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.825931 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.825671 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-data\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.825931 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.825693 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.825931 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.825719 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-token\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.825931 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.825783 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-credential-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.825931 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.825886 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-workload-certs\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.826188 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.825957 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-workload-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.826188 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.826015 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-envoy\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.926956 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.926876 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-workload-certs\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927133 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.926989 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-workload-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927133 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927028 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-envoy\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927204 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927167 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927248 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2lfb\" (UniqueName: \"kubernetes.io/projected/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-kube-api-access-c2lfb\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927288 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927261 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-data\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927288 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927281 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927386 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927344 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-workload-certs\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927436 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927382 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-workload-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927485 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927466 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-token\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927537 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927508 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-credential-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.927656 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927609 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-data\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.928012 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.927995 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.928106 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.928084 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-credential-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.929589 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.929567 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-envoy\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.929674 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.929638 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.935615 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.935595 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2lfb\" (UniqueName: \"kubernetes.io/projected/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-kube-api-access-c2lfb\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:16.935770 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:16.935732 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf64b0c-84b7-4d8a-8c4d-a877308ff994-istio-token\") pod \"maas-default-gateway-openshift-default-c747fcbf8-bbk2x\" (UID: \"0cf64b0c-84b7-4d8a-8c4d-a877308ff994\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:17.048871 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:17.048791 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:17.162324 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:17.162296 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:17.186560 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:17.186534 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x"] Aug 04 20:50:17.188773 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:50:17.188743 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf64b0c_84b7_4d8a_8c4d_a877308ff994.slice/crio-013978e0fe48f816fba921b2cd96e995c92e2fb5fdcf145a72d448304f970c4d WatchSource:0}: Error finding container 013978e0fe48f816fba921b2cd96e995c92e2fb5fdcf145a72d448304f970c4d: Status 404 returned error can't find the container with id 013978e0fe48f816fba921b2cd96e995c92e2fb5fdcf145a72d448304f970c4d Aug 04 20:50:17.190814 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:17.190786 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:50:17.190877 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:17.190847 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:50:17.190947 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:17.190876 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:50:17.678455 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:17.678423 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" event={"ID":"0cf64b0c-84b7-4d8a-8c4d-a877308ff994","Type":"ContainerStarted","Data":"225bce47ec68199df5df5bb0a2ab5e2a7eae79043bca191178065ec4811f4114"} Aug 04 20:50:17.678638 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:17.678482 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" event={"ID":"0cf64b0c-84b7-4d8a-8c4d-a877308ff994","Type":"ContainerStarted","Data":"013978e0fe48f816fba921b2cd96e995c92e2fb5fdcf145a72d448304f970c4d"} Aug 04 20:50:17.702430 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:17.702381 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" podStartSLOduration=1.702365173 podStartE2EDuration="1.702365173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:50:17.7001262 +0000 UTC m=+551.846996781" watchObservedRunningTime="2026-08-04 20:50:17.702365173 +0000 UTC m=+551.849235772" Aug 04 20:50:18.049343 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:18.049252 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:18.053973 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:18.053951 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:18.671235 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:18.671200 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:18.683195 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:18.683168 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:18.684075 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:18.684055 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:50:18.776719 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:18.776688 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:18.840081 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:18.840041 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:19.499662 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:19.499626 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:19.588813 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:19.588781 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:20.036745 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:20.036708 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:20.101213 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:20.101179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:27.165269 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:27.165233 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:27.227583 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:27.227550 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:34.759372 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:34.759339 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:34.859843 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:34.859812 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:35.271056 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:35.271024 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:47.006955 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:47.006900 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:47.574685 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:47.574650 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:53.405264 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:53.405084 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:53.498077 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:53.498041 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:53.579066 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:53.579030 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:53.642121 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:53.642093 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:56.749627 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:56.749597 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:50:56.848529 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:50:56.848500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:30.788584 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:30.788497 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:30.897288 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:30.897258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:32.497667 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:32.497631 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:32.601188 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:32.601157 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:32.999299 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:32.999265 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:33.110872 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:33.110837 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:41.969022 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:41.968979 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:43.048723 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:43.048684 2576 generic.go:356] "Generic (PLEG): container finished" podID="0cf64b0c-84b7-4d8a-8c4d-a877308ff994" containerID="225bce47ec68199df5df5bb0a2ab5e2a7eae79043bca191178065ec4811f4114" exitCode=0 Aug 04 20:51:43.049144 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:43.048751 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" event={"ID":"0cf64b0c-84b7-4d8a-8c4d-a877308ff994","Type":"ContainerDied","Data":"225bce47ec68199df5df5bb0a2ab5e2a7eae79043bca191178065ec4811f4114"} Aug 04 20:51:43.049144 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:43.049133 2576 scope.go:117] "RemoveContainer" containerID="225bce47ec68199df5df5bb0a2ab5e2a7eae79043bca191178065ec4811f4114" Aug 04 20:51:43.050261 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:43.050234 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:51:43.050335 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:43.050318 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:51:43.050394 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:43.050378 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 20:51:43.221203 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:43.221175 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:44.057304 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:44.057268 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" event={"ID":"0cf64b0c-84b7-4d8a-8c4d-a877308ff994","Type":"ContainerStarted","Data":"558b76e75cbcb8690a4967763d8033a73183a7b4996fd64d9315cdfdc5f22df1"} Aug 04 20:51:45.051036 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.050992 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:51:45.051253 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.051044 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-bbk2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Aug 04 20:51:45.051253 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.051107 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" podUID="0cf64b0c-84b7-4d8a-8c4d-a877308ff994" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Aug 04 20:51:45.360606 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.359871 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx"] Aug 04 20:51:45.362779 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.362758 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.365656 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.365635 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 20:51:45.366414 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.366395 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kftb7\"" Aug 04 20:51:45.366509 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.366395 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 20:51:45.366509 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.366394 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Aug 04 20:51:45.373802 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.373782 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx"] Aug 04 20:51:45.379713 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.379693 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:45.479511 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.479473 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.479511 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.479513 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.479761 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.479540 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.479761 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.479700 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.479761 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.479719 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9x99\" (UniqueName: \"kubernetes.io/projected/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-kube-api-access-m9x99\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.479761 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.479752 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.580962 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.580897 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.581155 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.581006 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.581155 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.581025 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.581155 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.581050 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.581155 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.581069 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.581155 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.581088 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9x99\" (UniqueName: \"kubernetes.io/projected/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-kube-api-access-m9x99\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.581454 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.581433 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.581521 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.581465 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.581640 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.581512 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.583550 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.583530 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.583706 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.583691 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.589079 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.589058 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9x99\" (UniqueName: \"kubernetes.io/projected/aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9-kube-api-access-m9x99\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx\" (UID: \"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.673941 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.673832 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:45.852927 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:45.851840 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx"] Aug 04 20:51:45.853661 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:51:45.853633 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeedafd0_92d3_41e6_b256_a4aa0f3cb1f9.slice/crio-a93526a7b4059fcc9a1f68824ee1c9c2495407757a9038ad164713a5b6662bac WatchSource:0}: Error finding container a93526a7b4059fcc9a1f68824ee1c9c2495407757a9038ad164713a5b6662bac: Status 404 returned error can't find the container with id a93526a7b4059fcc9a1f68824ee1c9c2495407757a9038ad164713a5b6662bac Aug 04 20:51:46.050105 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:46.049547 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-bbk2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Aug 04 20:51:46.050105 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:46.049607 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" podUID="0cf64b0c-84b7-4d8a-8c4d-a877308ff994" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Aug 04 20:51:46.068967 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:46.068935 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" event={"ID":"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9","Type":"ContainerStarted","Data":"a93526a7b4059fcc9a1f68824ee1c9c2495407757a9038ad164713a5b6662bac"} Aug 04 20:51:47.051425 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:47.051381 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-bbk2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Aug 04 20:51:47.051966 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:47.051450 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" podUID="0cf64b0c-84b7-4d8a-8c4d-a877308ff994" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Aug 04 20:51:48.050284 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:48.050237 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-bbk2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Aug 04 20:51:48.050496 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:48.050310 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" podUID="0cf64b0c-84b7-4d8a-8c4d-a877308ff994" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Aug 04 20:51:48.080621 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:48.080584 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" event={"ID":"aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9","Type":"ContainerStarted","Data":"db5de259595437a338981d4e9aac715c88ca456a7e96a56e8cc1e4158906836d"} Aug 04 20:51:48.102266 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:48.102217 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" podStartSLOduration=1.801060524 podStartE2EDuration="3.102200705s" totalImagesPullingTime="1.301140181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:51:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:51:48.101251862 +0000 UTC m=+642.248122432" watchObservedRunningTime="2026-08-04 20:51:48.102200705 +0000 UTC m=+642.249071259" Aug 04 20:51:48.684054 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:48.683998 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:51:49.049786 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:49.049684 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-bbk2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Aug 04 20:51:49.050098 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:49.050064 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" podUID="0cf64b0c-84b7-4d8a-8c4d-a877308ff994" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Aug 04 20:51:50.053482 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:50.053450 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:51:50.054482 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:50.054461 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-bbk2x" Aug 04 20:51:55.363634 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:55.363601 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:55.473676 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:55.473645 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:51:55.674207 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:55.674111 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:55.674207 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:55.674188 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:55.686963 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:55.686931 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:51:56.126855 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:51:56.126828 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx" Aug 04 20:52:04.659995 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:04.659958 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:52:04.760833 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:04.760796 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:52:06.651530 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.651495 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg"] Aug 04 20:52:06.656068 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.656043 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.666788 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.666751 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Aug 04 20:52:06.667722 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.667696 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg"] Aug 04 20:52:06.690019 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.689987 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:52:06.780543 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.780496 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.780738 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.780561 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.780738 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.780635 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.780738 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.780662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.780738 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.780680 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0de5437-789c-4d28-bc0a-1c89def6a0f3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.780955 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.780759 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgwdj\" (UniqueName: \"kubernetes.io/projected/b0de5437-789c-4d28-bc0a-1c89def6a0f3-kube-api-access-kgwdj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.881565 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.881523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.881767 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.881580 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.881767 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.881621 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.881767 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.881644 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.881767 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.881670 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0de5437-789c-4d28-bc0a-1c89def6a0f3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.882014 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.881837 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgwdj\" (UniqueName: \"kubernetes.io/projected/b0de5437-789c-4d28-bc0a-1c89def6a0f3-kube-api-access-kgwdj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.882076 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.882050 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.882127 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.882073 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.882203 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.882182 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.884200 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.884172 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0de5437-789c-4d28-bc0a-1c89def6a0f3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.884426 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.884409 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0de5437-789c-4d28-bc0a-1c89def6a0f3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.890343 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.890312 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgwdj\" (UniqueName: \"kubernetes.io/projected/b0de5437-789c-4d28-bc0a-1c89def6a0f3-kube-api-access-kgwdj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s5lg\" (UID: \"b0de5437-789c-4d28-bc0a-1c89def6a0f3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:06.967939 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:06.967805 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:07.103584 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:07.103549 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg"] Aug 04 20:52:07.106100 ip-10-0-139-54 kubenswrapper[2576]: W0804 20:52:07.106075 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0de5437_789c_4d28_bc0a_1c89def6a0f3.slice/crio-60168509cad9a1e1cc0fb4013233d3aff8429814bf2e280190bdf9749a6d8d80 WatchSource:0}: Error finding container 60168509cad9a1e1cc0fb4013233d3aff8429814bf2e280190bdf9749a6d8d80: Status 404 returned error can't find the container with id 60168509cad9a1e1cc0fb4013233d3aff8429814bf2e280190bdf9749a6d8d80 Aug 04 20:52:07.164573 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:07.164539 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" event={"ID":"b0de5437-789c-4d28-bc0a-1c89def6a0f3","Type":"ContainerStarted","Data":"60168509cad9a1e1cc0fb4013233d3aff8429814bf2e280190bdf9749a6d8d80"} Aug 04 20:52:08.170681 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:08.170641 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" event={"ID":"b0de5437-789c-4d28-bc0a-1c89def6a0f3","Type":"ContainerStarted","Data":"93c6027ee9669319e047b310b46ca0fe894cac2a4694d6459bc34e43f0258f7f"} Aug 04 20:52:08.197348 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:08.197297 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" podStartSLOduration=2.018971458 podStartE2EDuration="2.197280203s" totalImagesPullingTime="178.308745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:52:08.196097397 +0000 UTC m=+662.342967952" watchObservedRunningTime="2026-08-04 20:52:08.197280203 +0000 UTC m=+662.344150757" Aug 04 20:52:12.484362 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:12.484328 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:52:16.968447 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:16.968399 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:16.968893 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:16.968471 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:16.980774 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:16.980749 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:17.220572 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:17.220484 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s5lg" Aug 04 20:52:23.785115 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:23.785076 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:52:30.688029 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:52:30.688000 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:53:21.079082 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:53:21.079000 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:53:21.179143 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:53:21.179102 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:53:22.240862 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:53:22.240828 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:53:22.331744 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:53:22.331714 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:53:37.702076 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:53:37.702038 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:54:23.068889 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:54:23.068853 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:54:23.195587 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:54:23.195553 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:55:27.658048 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:55:27.657990 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:55:28.897828 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:55:28.897789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:55:44.488360 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:55:44.488324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:01.118086 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:01.118003 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:32.665819 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:32.665787 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:32.771785 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:32.771749 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:32.968837 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:32.968750 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:38.828756 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:38.828721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:38.932102 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:38.932069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:39.030486 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:39.030450 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:39.231933 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:39.231844 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:45.320854 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:45.320822 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:56:46.470457 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:56:46.470420 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 20:59:44.162094 ip-10-0-139-54 kubenswrapper[2576]: I0804 20:59:44.162055 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:00:24.577023 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:00:24.576986 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:00:24.676899 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:00:24.676862 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:00:29.659791 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:00:29.659704 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:00:29.754048 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:00:29.754013 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:03:26.038876 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:03:26.038843 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:06:34.225206 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:06:34.225125 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:06:50.140806 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:06:50.140774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:07:12.346008 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:07:12.345942 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:07:13.191142 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:07:13.191099 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:07:18.599187 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:07:18.599152 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:07:36.944196 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:07:36.944157 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:10:37.999492 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:10:37.999450 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:03.922918 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:03.922863 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:06.961663 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:06.961625 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:07.061244 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:07.061211 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:07.420889 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:07.420857 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:10.745531 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:10.745492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:10.845691 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:10.845652 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:13.390724 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:13.390683 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:13.451395 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:13.451365 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:13.681472 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:13.681384 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:13.943707 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:13.943614 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5"] Aug 04 21:11:13.947179 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:13.947155 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:13.950122 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:13.950102 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5a7758b8-openshift-default-dockercfg-td88x\"" Aug 04 21:11:13.968027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:13.968003 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5"] Aug 04 21:11:13.970822 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:13.970796 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:14.006028 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.006000 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-credential-socket\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.006155 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.006051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd8b364-bb29-49fb-8248-06f3c962c45b-istiod-ca-cert\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.006155 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.006136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-certs\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.006268 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.006194 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-token\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.006268 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.006241 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-socket\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.006348 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.006277 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9crzx\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-kube-api-access-9crzx\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.006348 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.006319 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-data\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.006413 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.006345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-envoy\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.006413 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.006390 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-podinfo\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.107890 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.107846 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9crzx\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-kube-api-access-9crzx\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108129 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.107931 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-data\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108129 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.107957 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-envoy\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108129 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108087 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-podinfo\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108303 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108145 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-credential-socket\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108303 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108186 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd8b364-bb29-49fb-8248-06f3c962c45b-istiod-ca-cert\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108303 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108214 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-certs\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108450 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108363 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-token\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108450 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108428 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-socket\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108556 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108460 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-data\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108556 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108502 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-credential-socket\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108652 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108550 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-certs\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108948 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108929 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd8b364-bb29-49fb-8248-06f3c962c45b-istiod-ca-cert\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.108948 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.108941 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-socket\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.110467 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.110442 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-envoy\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.110674 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.110660 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-podinfo\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.119614 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.119588 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-token\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.119763 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.119745 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9crzx\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-kube-api-access-9crzx\") pod \"e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.163954 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.163895 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:14.257811 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.257705 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:14.395555 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.395531 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5"] Aug 04 21:11:14.399277 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:11:14.399246 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd8b364_bb29_49fb_8248_06f3c962c45b.slice/crio-a6462d49dda7e965b7dae61418ffb4eaa8f97d7dcdaf0fabde675c9fe6e95c86 WatchSource:0}: Error finding container a6462d49dda7e965b7dae61418ffb4eaa8f97d7dcdaf0fabde675c9fe6e95c86: Status 404 returned error can't find the container with id a6462d49dda7e965b7dae61418ffb4eaa8f97d7dcdaf0fabde675c9fe6e95c86 Aug 04 21:11:14.401542 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.401508 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:11:14.401630 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.401578 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:11:14.401630 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:14.401607 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:11:15.211542 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:15.211502 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" event={"ID":"6fd8b364-bb29-49fb-8248-06f3c962c45b","Type":"ContainerStarted","Data":"dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da"} Aug 04 21:11:15.211745 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:15.211600 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" event={"ID":"6fd8b364-bb29-49fb-8248-06f3c962c45b","Type":"ContainerStarted","Data":"a6462d49dda7e965b7dae61418ffb4eaa8f97d7dcdaf0fabde675c9fe6e95c86"} Aug 04 21:11:15.233644 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:15.233595 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" podStartSLOduration=2.2335794079999998 podStartE2EDuration="2.233579408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:11:15.232575535 +0000 UTC m=+1809.379446088" watchObservedRunningTime="2026-08-04 21:11:15.233579408 +0000 UTC m=+1809.380449962" Aug 04 21:11:15.258842 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:15.258808 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:15.263745 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:15.263714 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:16.216240 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:16.216207 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:16.217184 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:16.217161 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:16.579548 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:16.579460 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:16.679325 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:16.679290 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:17.722387 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:17.722348 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:18.070471 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:18.070381 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:18.271306 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:18.271273 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:18.372301 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:18.372264 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:30.284433 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:30.284396 2576 generic.go:356] "Generic (PLEG): container finished" podID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerID="dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da" exitCode=0 Aug 04 21:11:30.284840 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:30.284470 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" event={"ID":"6fd8b364-bb29-49fb-8248-06f3c962c45b","Type":"ContainerDied","Data":"dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da"} Aug 04 21:11:30.284884 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:30.284854 2576 scope.go:117] "RemoveContainer" containerID="dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da" Aug 04 21:11:30.285793 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:30.285769 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:11:30.285853 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:30.285842 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:11:30.285894 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:30.285876 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:11:31.216711 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:31.216674 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:31.291138 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:31.291097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" event={"ID":"6fd8b364-bb29-49fb-8248-06f3c962c45b","Type":"ContainerStarted","Data":"cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b"} Aug 04 21:11:32.258635 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:32.258599 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:32.259535 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:32.259509 2576 patch_prober.go:28] interesting pod/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Aug 04 21:11:32.259643 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:32.259572 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Aug 04 21:11:33.258690 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:33.258649 2576 patch_prober.go:28] interesting pod/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Aug 04 21:11:33.259124 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:33.258729 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Aug 04 21:11:34.258704 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:34.258667 2576 patch_prober.go:28] interesting pod/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Aug 04 21:11:34.259119 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:34.258739 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Aug 04 21:11:35.258711 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:35.258667 2576 patch_prober.go:28] interesting pod/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Aug 04 21:11:35.259185 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:35.258731 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Aug 04 21:11:36.258771 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:36.258731 2576 patch_prober.go:28] interesting pod/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Aug 04 21:11:36.259190 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:36.258800 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Aug 04 21:11:37.261556 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:37.261519 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:37.262045 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:37.261809 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:37.262435 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:37.262418 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:42.156619 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.156579 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r"] Aug 04 21:11:42.159153 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.159127 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" Aug 04 21:11:42.162497 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.162464 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 21:11:42.162497 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.162486 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86svs\"" Aug 04 21:11:42.170038 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.170015 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r"] Aug 04 21:11:42.263150 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.263113 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c884bc2-9627-4f5d-bad0-8225e02c3059-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r\" (UID: \"5c884bc2-9627-4f5d-bad0-8225e02c3059\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" Aug 04 21:11:42.364133 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.364096 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c884bc2-9627-4f5d-bad0-8225e02c3059-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r\" (UID: \"5c884bc2-9627-4f5d-bad0-8225e02c3059\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" Aug 04 21:11:42.364705 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.364684 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c884bc2-9627-4f5d-bad0-8225e02c3059-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r\" (UID: \"5c884bc2-9627-4f5d-bad0-8225e02c3059\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" Aug 04 21:11:42.470221 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.470132 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" Aug 04 21:11:42.601350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.601322 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r"] Aug 04 21:11:42.603552 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:11:42.603525 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c884bc2_9627_4f5d_bad0_8225e02c3059.slice/crio-567b2a87fbff2fefa3661122ed32bca2f2fff790ebb3869150f27ce4fb477a4b WatchSource:0}: Error finding container 567b2a87fbff2fefa3661122ed32bca2f2fff790ebb3869150f27ce4fb477a4b: Status 404 returned error can't find the container with id 567b2a87fbff2fefa3661122ed32bca2f2fff790ebb3869150f27ce4fb477a4b Aug 04 21:11:42.605400 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:42.605381 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 21:11:43.351635 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:43.351587 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" event={"ID":"5c884bc2-9627-4f5d-bad0-8225e02c3059","Type":"ContainerStarted","Data":"567b2a87fbff2fefa3661122ed32bca2f2fff790ebb3869150f27ce4fb477a4b"} Aug 04 21:11:45.364224 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:45.364182 2576 generic.go:356] "Generic (PLEG): container finished" podID="5c884bc2-9627-4f5d-bad0-8225e02c3059" containerID="5cf66c93528cd7534af20b0b57f41b8ae89aee7f8abb04cd681deb05b8628476" exitCode=0 Aug 04 21:11:45.364667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:45.364231 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" event={"ID":"5c884bc2-9627-4f5d-bad0-8225e02c3059","Type":"ContainerDied","Data":"5cf66c93528cd7534af20b0b57f41b8ae89aee7f8abb04cd681deb05b8628476"} Aug 04 21:11:46.499982 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:46.499958 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" Aug 04 21:11:46.598145 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:46.598100 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c884bc2-9627-4f5d-bad0-8225e02c3059-maas-api-service-ca\") pod \"5c884bc2-9627-4f5d-bad0-8225e02c3059\" (UID: \"5c884bc2-9627-4f5d-bad0-8225e02c3059\") " Aug 04 21:11:46.598469 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:46.598448 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c884bc2-9627-4f5d-bad0-8225e02c3059-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5c884bc2-9627-4f5d-bad0-8225e02c3059" (UID: "5c884bc2-9627-4f5d-bad0-8225e02c3059"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:11:46.699817 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:46.699731 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c884bc2-9627-4f5d-bad0-8225e02c3059-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:47.380846 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:47.380805 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" event={"ID":"5c884bc2-9627-4f5d-bad0-8225e02c3059","Type":"ContainerDied","Data":"567b2a87fbff2fefa3661122ed32bca2f2fff790ebb3869150f27ce4fb477a4b"} Aug 04 21:11:47.380846 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:47.380841 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r" Aug 04 21:11:47.381134 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:47.380879 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="567b2a87fbff2fefa3661122ed32bca2f2fff790ebb3869150f27ce4fb477a4b" Aug 04 21:11:48.433455 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:48.433424 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:48.548424 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:48.548388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:48.650046 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:48.650007 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:48.769714 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:48.769606 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r"] Aug 04 21:11:48.775019 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:48.774984 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5a7758b8-drd9r"] Aug 04 21:11:48.778394 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:48.778368 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:48.919228 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:48.919194 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5"] Aug 04 21:11:48.919416 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:48.919396 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" containerID="cri-o://cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b" gracePeriod=30 Aug 04 21:11:48.974455 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:48.974418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:49.178435 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:49.178399 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:49.428576 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:49.428488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:49.527847 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:49.527816 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:49.645598 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:49.645561 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:50.031484 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.031450 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j"] Aug 04 21:11:50.031845 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.031831 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c884bc2-9627-4f5d-bad0-8225e02c3059" containerName="revoke-keys" Aug 04 21:11:50.031885 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.031846 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c884bc2-9627-4f5d-bad0-8225e02c3059" containerName="revoke-keys" Aug 04 21:11:50.031940 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.031900 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c884bc2-9627-4f5d-bad0-8225e02c3059" containerName="revoke-keys" Aug 04 21:11:50.033957 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.033940 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.039517 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.039490 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:50.046667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.046650 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ad77b99e-gw-openshift-default-dockercfg-tpqh7\"" Aug 04 21:11:50.057593 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.057567 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j"] Aug 04 21:11:50.128077 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.128040 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-credential-socket\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.128260 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.128081 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb8w7\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-kube-api-access-hb8w7\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.128260 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.128157 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-socket\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.128260 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.128185 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-podinfo\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.128260 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.128211 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8e48f40-77e0-4e2f-a93f-129df46260e4-istiod-ca-cert\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.128260 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.128245 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-envoy\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.128437 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.128375 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-token\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.128437 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.128427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-data\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.128519 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.128495 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-certs\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.168220 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.168185 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:50.229798 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.229757 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-socket\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.229798 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.229795 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-podinfo\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.229833 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8e48f40-77e0-4e2f-a93f-129df46260e4-istiod-ca-cert\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.229866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-envoy\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.229955 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-token\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.229992 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-data\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230221 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.230036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-certs\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230221 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.230063 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-credential-socket\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230221 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.230087 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb8w7\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-kube-api-access-hb8w7\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230367 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.230267 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-socket\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230427 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.230383 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-data\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230543 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.230517 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-certs\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230657 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.230524 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-credential-socket\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.230760 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.230741 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8e48f40-77e0-4e2f-a93f-129df46260e4-istiod-ca-cert\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.232892 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.232872 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-envoy\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.233067 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.233049 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-podinfo\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.240033 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.240008 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-token\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.241619 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.241594 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb8w7\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-kube-api-access-hb8w7\") pod \"e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.344527 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.344497 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:50.442165 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.442118 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c884bc2-9627-4f5d-bad0-8225e02c3059" path="/var/lib/kubelet/pods/5c884bc2-9627-4f5d-bad0-8225e02c3059/volumes" Aug 04 21:11:50.516288 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.516259 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j"] Aug 04 21:11:50.518162 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:11:50.518136 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e48f40_77e0_4e2f_a93f_129df46260e4.slice/crio-9b8ca02076ac3e27decea429aa51725af2311576878d87a8cbf805b8d39b0961 WatchSource:0}: Error finding container 9b8ca02076ac3e27decea429aa51725af2311576878d87a8cbf805b8d39b0961: Status 404 returned error can't find the container with id 9b8ca02076ac3e27decea429aa51725af2311576878d87a8cbf805b8d39b0961 Aug 04 21:11:50.520456 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.520427 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:11:50.520509 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.520491 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:11:50.520551 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:50.520530 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:11:51.401600 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:51.401565 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" event={"ID":"d8e48f40-77e0-4e2f-a93f-129df46260e4","Type":"ContainerStarted","Data":"8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958"} Aug 04 21:11:51.402037 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:51.401616 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" event={"ID":"d8e48f40-77e0-4e2f-a93f-129df46260e4","Type":"ContainerStarted","Data":"9b8ca02076ac3e27decea429aa51725af2311576878d87a8cbf805b8d39b0961"} Aug 04 21:11:51.437350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:51.437296 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" podStartSLOduration=2.437281376 podStartE2EDuration="2.437281376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:11:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:11:51.436203095 +0000 UTC m=+1845.583073651" watchObservedRunningTime="2026-08-04 21:11:51.437281376 +0000 UTC m=+1845.584151931" Aug 04 21:11:52.264111 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:52.264064 2576 patch_prober.go:28] interesting pod/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 21:11:52.264306 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:52.264151 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 21:11:52.345091 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:52.345047 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:52.349881 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:52.349854 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:52.406468 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:52.406421 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:52.407480 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:52.407461 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:11:52.964794 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:52.964755 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:53.184477 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:53.184442 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:53.434592 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:53.434560 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:53.787010 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:53.786930 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:53.898977 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:53.898932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:11:54.177582 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.177558 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:54.268461 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268423 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-certs\") pod \"6fd8b364-bb29-49fb-8248-06f3c962c45b\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " Aug 04 21:11:54.268461 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268460 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9crzx\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-kube-api-access-9crzx\") pod \"6fd8b364-bb29-49fb-8248-06f3c962c45b\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " Aug 04 21:11:54.268712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268483 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-credential-socket\") pod \"6fd8b364-bb29-49fb-8248-06f3c962c45b\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " Aug 04 21:11:54.268712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268524 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-podinfo\") pod \"6fd8b364-bb29-49fb-8248-06f3c962c45b\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " Aug 04 21:11:54.268712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268540 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-token\") pod \"6fd8b364-bb29-49fb-8248-06f3c962c45b\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " Aug 04 21:11:54.268712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268563 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-socket\") pod \"6fd8b364-bb29-49fb-8248-06f3c962c45b\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " Aug 04 21:11:54.268712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268626 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-data\") pod \"6fd8b364-bb29-49fb-8248-06f3c962c45b\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " Aug 04 21:11:54.268712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268652 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-envoy\") pod \"6fd8b364-bb29-49fb-8248-06f3c962c45b\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " Aug 04 21:11:54.268712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268694 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd8b364-bb29-49fb-8248-06f3c962c45b-istiod-ca-cert\") pod \"6fd8b364-bb29-49fb-8248-06f3c962c45b\" (UID: \"6fd8b364-bb29-49fb-8248-06f3c962c45b\") " Aug 04 21:11:54.269090 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268764 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6fd8b364-bb29-49fb-8248-06f3c962c45b" (UID: "6fd8b364-bb29-49fb-8248-06f3c962c45b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:11:54.269090 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268936 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6fd8b364-bb29-49fb-8248-06f3c962c45b" (UID: "6fd8b364-bb29-49fb-8248-06f3c962c45b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:11:54.269090 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.268984 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6fd8b364-bb29-49fb-8248-06f3c962c45b" (UID: "6fd8b364-bb29-49fb-8248-06f3c962c45b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:11:54.269432 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.269107 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:54.269432 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.269124 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:54.269432 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.269138 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:54.269432 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.269174 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-data" (OuterVolumeSpecName: "istio-data") pod "6fd8b364-bb29-49fb-8248-06f3c962c45b" (UID: "6fd8b364-bb29-49fb-8248-06f3c962c45b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:11:54.269432 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.269192 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd8b364-bb29-49fb-8248-06f3c962c45b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6fd8b364-bb29-49fb-8248-06f3c962c45b" (UID: "6fd8b364-bb29-49fb-8248-06f3c962c45b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:11:54.271215 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.271187 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6fd8b364-bb29-49fb-8248-06f3c962c45b" (UID: "6fd8b364-bb29-49fb-8248-06f3c962c45b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:11:54.271319 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.271245 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-kube-api-access-9crzx" (OuterVolumeSpecName: "kube-api-access-9crzx") pod "6fd8b364-bb29-49fb-8248-06f3c962c45b" (UID: "6fd8b364-bb29-49fb-8248-06f3c962c45b"). InnerVolumeSpecName "kube-api-access-9crzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:11:54.271509 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.271487 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-token" (OuterVolumeSpecName: "istio-token") pod "6fd8b364-bb29-49fb-8248-06f3c962c45b" (UID: "6fd8b364-bb29-49fb-8248-06f3c962c45b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:11:54.271619 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.271595 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6fd8b364-bb29-49fb-8248-06f3c962c45b" (UID: "6fd8b364-bb29-49fb-8248-06f3c962c45b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:11:54.370475 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.370439 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9crzx\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-kube-api-access-9crzx\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:54.370475 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.370473 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:54.370671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.370484 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:54.370671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.370491 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:54.370671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.370499 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd8b364-bb29-49fb-8248-06f3c962c45b-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:54.370671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.370506 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd8b364-bb29-49fb-8248-06f3c962c45b-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:11:54.416955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.416891 2576 generic.go:356] "Generic (PLEG): container finished" podID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerID="cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b" exitCode=0 Aug 04 21:11:54.416955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.416945 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" event={"ID":"6fd8b364-bb29-49fb-8248-06f3c962c45b","Type":"ContainerDied","Data":"cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b"} Aug 04 21:11:54.417134 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.416967 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" Aug 04 21:11:54.417134 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.417006 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5" event={"ID":"6fd8b364-bb29-49fb-8248-06f3c962c45b","Type":"ContainerDied","Data":"a6462d49dda7e965b7dae61418ffb4eaa8f97d7dcdaf0fabde675c9fe6e95c86"} Aug 04 21:11:54.417134 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.417030 2576 scope.go:117] "RemoveContainer" containerID="cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b" Aug 04 21:11:54.426951 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.426931 2576 scope.go:117] "RemoveContainer" containerID="dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da" Aug 04 21:11:54.435525 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.435505 2576 scope.go:117] "RemoveContainer" containerID="cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b" Aug 04 21:11:54.435877 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:11:54.435784 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b\": container with ID starting with cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b not found: ID does not exist" containerID="cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b" Aug 04 21:11:54.435877 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.435811 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b"} err="failed to get container status \"cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b\": rpc error: code = NotFound desc = could not find container \"cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b\": container with ID starting with cccf1d89dc3be0de76665142d6fd3881401988eb5fbfe857556663084cb91e2b not found: ID does not exist" Aug 04 21:11:54.435877 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.435827 2576 scope.go:117] "RemoveContainer" containerID="dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da" Aug 04 21:11:54.436098 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:11:54.436078 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da\": container with ID starting with dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da not found: ID does not exist" containerID="dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da" Aug 04 21:11:54.436213 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.436104 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da"} err="failed to get container status \"dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da\": rpc error: code = NotFound desc = could not find container \"dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da\": container with ID starting with dc258888ace08c10298dd5fbdae99745931a5fc8dba76c529cbc5355391c56da not found: ID does not exist" Aug 04 21:11:54.441492 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.441466 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5"] Aug 04 21:11:54.445807 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:54.445785 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5a7758b8-openshift-default-5d5db5cdb-2xxb5"] Aug 04 21:11:56.440375 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:11:56.440343 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" path="/var/lib/kubelet/pods/6fd8b364-bb29-49fb-8248-06f3c962c45b/volumes" Aug 04 21:12:05.482262 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:05.482182 2576 generic.go:356] "Generic (PLEG): container finished" podID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerID="8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958" exitCode=0 Aug 04 21:12:05.482657 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:05.482258 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" event={"ID":"d8e48f40-77e0-4e2f-a93f-129df46260e4","Type":"ContainerDied","Data":"8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958"} Aug 04 21:12:05.482657 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:05.482636 2576 scope.go:117] "RemoveContainer" containerID="8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958" Aug 04 21:12:05.483639 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:05.483612 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:12:05.483713 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:05.483696 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:12:05.483762 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:05.483746 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:12:06.488311 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:06.488277 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" event={"ID":"d8e48f40-77e0-4e2f-a93f-129df46260e4","Type":"ContainerStarted","Data":"9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b"} Aug 04 21:12:07.345150 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:07.345107 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:12:07.346563 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:07.346531 2576 patch_prober.go:28] interesting pod/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Aug 04 21:12:07.347093 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:07.346803 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Aug 04 21:12:07.407351 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:07.407310 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:12:08.346096 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:08.345573 2576 patch_prober.go:28] interesting pod/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Aug 04 21:12:08.346096 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:08.345656 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Aug 04 21:12:09.345405 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:09.345357 2576 patch_prober.go:28] interesting pod/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Aug 04 21:12:09.345719 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:09.345679 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Aug 04 21:12:10.345049 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:10.345010 2576 patch_prober.go:28] interesting pod/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Aug 04 21:12:10.345667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:10.345632 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Aug 04 21:12:11.344997 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:11.344955 2576 patch_prober.go:28] interesting pod/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Aug 04 21:12:11.345286 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:11.345257 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Aug 04 21:12:12.348866 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:12.348832 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:12:12.349900 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:12.349876 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:12:17.903612 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.903552 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7"] Aug 04 21:12:17.904040 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.903882 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" Aug 04 21:12:17.904040 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.903893 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" Aug 04 21:12:17.904040 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.903918 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" Aug 04 21:12:17.904040 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.903924 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" Aug 04 21:12:17.904040 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.903986 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" Aug 04 21:12:17.904040 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.903994 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd8b364-bb29-49fb-8248-06f3c962c45b" containerName="istio-proxy" Aug 04 21:12:17.909178 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.909159 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" Aug 04 21:12:17.912316 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.912141 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 21:12:17.913154 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.913128 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86svs\"" Aug 04 21:12:17.914506 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:17.914482 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7"] Aug 04 21:12:18.002236 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:18.002205 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1e5bb2-de3b-4b9c-a023-a92f27de4190-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7\" (UID: \"ad1e5bb2-de3b-4b9c-a023-a92f27de4190\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" Aug 04 21:12:18.103211 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:18.103172 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1e5bb2-de3b-4b9c-a023-a92f27de4190-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7\" (UID: \"ad1e5bb2-de3b-4b9c-a023-a92f27de4190\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" Aug 04 21:12:18.103833 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:18.103815 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1e5bb2-de3b-4b9c-a023-a92f27de4190-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7\" (UID: \"ad1e5bb2-de3b-4b9c-a023-a92f27de4190\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" Aug 04 21:12:18.221834 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:18.221742 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" Aug 04 21:12:18.353631 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:18.353478 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7"] Aug 04 21:12:18.356185 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:12:18.356155 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad1e5bb2_de3b_4b9c_a023_a92f27de4190.slice/crio-0f0fb253d9ae49ebad797f1b075d343182b1bc145b5fffc8add77139a934c9bc WatchSource:0}: Error finding container 0f0fb253d9ae49ebad797f1b075d343182b1bc145b5fffc8add77139a934c9bc: Status 404 returned error can't find the container with id 0f0fb253d9ae49ebad797f1b075d343182b1bc145b5fffc8add77139a934c9bc Aug 04 21:12:18.545761 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:18.545726 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" event={"ID":"ad1e5bb2-de3b-4b9c-a023-a92f27de4190","Type":"ContainerStarted","Data":"0f0fb253d9ae49ebad797f1b075d343182b1bc145b5fffc8add77139a934c9bc"} Aug 04 21:12:19.554919 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:19.554873 2576 generic.go:356] "Generic (PLEG): container finished" podID="ad1e5bb2-de3b-4b9c-a023-a92f27de4190" containerID="a987e402f74edf5cf61e94e48c87faa5999c0e31c8171d6016460f5647c7ee28" exitCode=0 Aug 04 21:12:19.555344 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:19.554934 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" event={"ID":"ad1e5bb2-de3b-4b9c-a023-a92f27de4190","Type":"ContainerDied","Data":"a987e402f74edf5cf61e94e48c87faa5999c0e31c8171d6016460f5647c7ee28"} Aug 04 21:12:20.691798 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:20.691773 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" Aug 04 21:12:20.827299 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:20.827197 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1e5bb2-de3b-4b9c-a023-a92f27de4190-maas-api-service-ca\") pod \"ad1e5bb2-de3b-4b9c-a023-a92f27de4190\" (UID: \"ad1e5bb2-de3b-4b9c-a023-a92f27de4190\") " Aug 04 21:12:20.827574 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:20.827549 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad1e5bb2-de3b-4b9c-a023-a92f27de4190-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ad1e5bb2-de3b-4b9c-a023-a92f27de4190" (UID: "ad1e5bb2-de3b-4b9c-a023-a92f27de4190"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:12:20.928129 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:20.928091 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1e5bb2-de3b-4b9c-a023-a92f27de4190-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:21.566037 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:21.566006 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" Aug 04 21:12:21.566220 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:21.566037 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7" event={"ID":"ad1e5bb2-de3b-4b9c-a023-a92f27de4190","Type":"ContainerDied","Data":"0f0fb253d9ae49ebad797f1b075d343182b1bc145b5fffc8add77139a934c9bc"} Aug 04 21:12:21.566220 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:21.566092 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f0fb253d9ae49ebad797f1b075d343182b1bc145b5fffc8add77139a934c9bc" Aug 04 21:12:24.528717 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:24.528661 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:24.603408 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:24.603365 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:24.661598 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:24.661562 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:24.696262 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:24.696226 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7"] Aug 04 21:12:24.699069 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:24.699046 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad77b99e-nf4r7"] Aug 04 21:12:24.716789 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:24.716760 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:24.834568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:24.834470 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j"] Aug 04 21:12:24.834740 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:24.834695 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" containerID="cri-o://9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b" gracePeriod=30 Aug 04 21:12:24.857473 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:24.857429 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:25.077410 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.077366 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:25.307163 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.307123 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:25.619618 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.619587 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79"] Aug 04 21:12:25.620126 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.620106 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad1e5bb2-de3b-4b9c-a023-a92f27de4190" containerName="revoke-keys" Aug 04 21:12:25.620203 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.620130 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad1e5bb2-de3b-4b9c-a023-a92f27de4190" containerName="revoke-keys" Aug 04 21:12:25.620256 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.620225 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad1e5bb2-de3b-4b9c-a023-a92f27de4190" containerName="revoke-keys" Aug 04 21:12:25.623404 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.623382 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.626387 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.626362 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-213b68bd-openshift-default-dockercfg-6gw7t\"" Aug 04 21:12:25.635451 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.635426 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79"] Aug 04 21:12:25.646086 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.646059 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:25.737332 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.737293 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:25.773499 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.773469 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsq7c\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-kube-api-access-dsq7c\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.773674 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.773503 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-data\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.773674 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.773524 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istiod-ca-cert\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.773674 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.773548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-credential-socket\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.773674 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.773618 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-envoy\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.773833 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.773674 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-token\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.773833 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.773702 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-socket\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.773833 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.773720 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-podinfo\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.773833 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.773738 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-certs\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.875376 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875279 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-credential-socket\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.875376 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875328 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-envoy\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.875376 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-token\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.875671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875384 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-socket\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.875671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875403 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-podinfo\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.875671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875421 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-certs\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.875671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875506 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsq7c\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-kube-api-access-dsq7c\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.875671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-data\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.875671 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875562 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istiod-ca-cert\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.876001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875752 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-credential-socket\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.876001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875806 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-socket\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.876001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.875898 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-certs\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.876105 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.876023 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-data\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.876317 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.876298 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istiod-ca-cert\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.878078 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.878055 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-envoy\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.878194 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.878168 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-podinfo\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.900218 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.900196 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-token\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.900340 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.900282 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsq7c\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-kube-api-access-dsq7c\") pod \"e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:25.936435 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:25.936400 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:26.086344 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:26.086307 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79"] Aug 04 21:12:26.089324 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:12:26.089294 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ea91a6_3e5a_4fb4_aa22_6ac8642ca1b2.slice/crio-abe3121ea0b9ec762a90b4d7860f37d8a20df1d50d33d8640312acf80e21704a WatchSource:0}: Error finding container abe3121ea0b9ec762a90b4d7860f37d8a20df1d50d33d8640312acf80e21704a: Status 404 returned error can't find the container with id abe3121ea0b9ec762a90b4d7860f37d8a20df1d50d33d8640312acf80e21704a Aug 04 21:12:26.091444 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:26.091407 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:12:26.091556 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:26.091477 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:12:26.091556 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:26.091509 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:12:26.440466 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:26.440369 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad1e5bb2-de3b-4b9c-a023-a92f27de4190" path="/var/lib/kubelet/pods/ad1e5bb2-de3b-4b9c-a023-a92f27de4190/volumes" Aug 04 21:12:26.591170 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:26.591133 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" event={"ID":"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2","Type":"ContainerStarted","Data":"a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c"} Aug 04 21:12:26.591321 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:26.591187 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" event={"ID":"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2","Type":"ContainerStarted","Data":"abe3121ea0b9ec762a90b4d7860f37d8a20df1d50d33d8640312acf80e21704a"} Aug 04 21:12:26.614169 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:26.614119 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" podStartSLOduration=1.6141038330000002 podStartE2EDuration="1.614103833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:12:26.612738231 +0000 UTC m=+1880.759608796" watchObservedRunningTime="2026-08-04 21:12:26.614103833 +0000 UTC m=+1880.760974445" Aug 04 21:12:26.936804 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:26.936765 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:27.351675 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:27.351639 2576 patch_prober.go:28] interesting pod/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Aug 04 21:12:27.351848 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:27.351698 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 21:12:27.942238 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:27.942211 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:28.049335 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:28.049301 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:28.148308 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:28.148271 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:28.601352 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:28.601319 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:28.602452 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:28.602431 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:29.179835 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:29.179791 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:29.440286 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:29.440190 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:29.704866 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:29.704781 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:29.804071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:29.804042 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:30.081797 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.081771 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:12:30.215493 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215452 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8e48f40-77e0-4e2f-a93f-129df46260e4-istiod-ca-cert\") pod \"d8e48f40-77e0-4e2f-a93f-129df46260e4\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " Aug 04 21:12:30.215493 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215505 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-credential-socket\") pod \"d8e48f40-77e0-4e2f-a93f-129df46260e4\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215554 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-socket\") pod \"d8e48f40-77e0-4e2f-a93f-129df46260e4\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215578 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-certs\") pod \"d8e48f40-77e0-4e2f-a93f-129df46260e4\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215603 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-envoy\") pod \"d8e48f40-77e0-4e2f-a93f-129df46260e4\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215627 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-token\") pod \"d8e48f40-77e0-4e2f-a93f-129df46260e4\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215690 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-podinfo\") pod \"d8e48f40-77e0-4e2f-a93f-129df46260e4\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215715 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-data\") pod \"d8e48f40-77e0-4e2f-a93f-129df46260e4\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215748 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb8w7\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-kube-api-access-hb8w7\") pod \"d8e48f40-77e0-4e2f-a93f-129df46260e4\" (UID: \"d8e48f40-77e0-4e2f-a93f-129df46260e4\") " Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215884 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d8e48f40-77e0-4e2f-a93f-129df46260e4" (UID: "d8e48f40-77e0-4e2f-a93f-129df46260e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215948 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d8e48f40-77e0-4e2f-a93f-129df46260e4" (UID: "d8e48f40-77e0-4e2f-a93f-129df46260e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215983 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d8e48f40-77e0-4e2f-a93f-129df46260e4" (UID: "d8e48f40-77e0-4e2f-a93f-129df46260e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:12:30.216071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.215990 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8e48f40-77e0-4e2f-a93f-129df46260e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d8e48f40-77e0-4e2f-a93f-129df46260e4" (UID: "d8e48f40-77e0-4e2f-a93f-129df46260e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:12:30.217016 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.216098 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "d8e48f40-77e0-4e2f-a93f-129df46260e4" (UID: "d8e48f40-77e0-4e2f-a93f-129df46260e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:12:30.217016 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.216241 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:30.217016 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.216258 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8e48f40-77e0-4e2f-a93f-129df46260e4-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:30.217016 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.216270 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:30.217016 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.216278 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:30.217016 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.216286 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:30.218273 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.218242 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "d8e48f40-77e0-4e2f-a93f-129df46260e4" (UID: "d8e48f40-77e0-4e2f-a93f-129df46260e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:12:30.218445 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.218293 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d8e48f40-77e0-4e2f-a93f-129df46260e4" (UID: "d8e48f40-77e0-4e2f-a93f-129df46260e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:12:30.218445 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.218294 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-kube-api-access-hb8w7" (OuterVolumeSpecName: "kube-api-access-hb8w7") pod "d8e48f40-77e0-4e2f-a93f-129df46260e4" (UID: "d8e48f40-77e0-4e2f-a93f-129df46260e4"). InnerVolumeSpecName "kube-api-access-hb8w7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:12:30.218610 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.218589 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d8e48f40-77e0-4e2f-a93f-129df46260e4" (UID: "d8e48f40-77e0-4e2f-a93f-129df46260e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:12:30.317767 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.317666 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:30.317767 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.317706 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:30.317767 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.317717 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8e48f40-77e0-4e2f-a93f-129df46260e4-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:30.317767 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.317725 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb8w7\" (UniqueName: \"kubernetes.io/projected/d8e48f40-77e0-4e2f-a93f-129df46260e4-kube-api-access-hb8w7\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:30.612498 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.612457 2576 generic.go:356] "Generic (PLEG): container finished" podID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerID="9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b" exitCode=0 Aug 04 21:12:30.612667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.612509 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" Aug 04 21:12:30.612667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.612543 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" event={"ID":"d8e48f40-77e0-4e2f-a93f-129df46260e4","Type":"ContainerDied","Data":"9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b"} Aug 04 21:12:30.612667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.612598 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j" event={"ID":"d8e48f40-77e0-4e2f-a93f-129df46260e4","Type":"ContainerDied","Data":"9b8ca02076ac3e27decea429aa51725af2311576878d87a8cbf805b8d39b0961"} Aug 04 21:12:30.612667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.612627 2576 scope.go:117] "RemoveContainer" containerID="9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b" Aug 04 21:12:30.621575 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.621547 2576 scope.go:117] "RemoveContainer" containerID="8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958" Aug 04 21:12:30.629689 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.629670 2576 scope.go:117] "RemoveContainer" containerID="9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b" Aug 04 21:12:30.629992 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:12:30.629972 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b\": container with ID starting with 9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b not found: ID does not exist" containerID="9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b" Aug 04 21:12:30.630043 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.630003 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b"} err="failed to get container status \"9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b\": rpc error: code = NotFound desc = could not find container \"9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b\": container with ID starting with 9c13b5dc69556b53d12d45364b7984501c85b93a673af3e88c4220700820606b not found: ID does not exist" Aug 04 21:12:30.630043 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.630024 2576 scope.go:117] "RemoveContainer" containerID="8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958" Aug 04 21:12:30.630423 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:12:30.630393 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958\": container with ID starting with 8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958 not found: ID does not exist" containerID="8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958" Aug 04 21:12:30.631108 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.630432 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958"} err="failed to get container status \"8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958\": rpc error: code = NotFound desc = could not find container \"8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958\": container with ID starting with 8f0d3d111e5f83eb60070de59216ad929a962c3ad97c791b79e47cd4c025e958 not found: ID does not exist" Aug 04 21:12:30.633316 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.633290 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j"] Aug 04 21:12:30.634850 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:30.634826 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ad77b99e-gw-openshift-default-59744486b4-9vq8j"] Aug 04 21:12:32.440606 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:32.440558 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" path="/var/lib/kubelet/pods/d8e48f40-77e0-4e2f-a93f-129df46260e4/volumes" Aug 04 21:12:42.685680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:42.685643 2576 generic.go:356] "Generic (PLEG): container finished" podID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerID="a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c" exitCode=0 Aug 04 21:12:42.686175 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:42.685714 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" event={"ID":"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2","Type":"ContainerDied","Data":"a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c"} Aug 04 21:12:42.686175 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:42.686131 2576 scope.go:117] "RemoveContainer" containerID="a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c" Aug 04 21:12:42.687111 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:42.687087 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:12:42.687173 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:42.687152 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:12:42.687210 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:42.687187 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:12:43.601977 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:43.601940 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:43.692545 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:43.692511 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" event={"ID":"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2","Type":"ContainerStarted","Data":"ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604"} Aug 04 21:12:43.936948 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:43.936827 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:43.938438 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:43.938409 2576 patch_prober.go:28] interesting pod/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Aug 04 21:12:43.938696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:43.938661 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Aug 04 21:12:44.937724 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:44.937673 2576 patch_prober.go:28] interesting pod/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Aug 04 21:12:44.938297 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:44.937747 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Aug 04 21:12:45.937816 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:45.937775 2576 patch_prober.go:28] interesting pod/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Aug 04 21:12:45.938478 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:45.938441 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Aug 04 21:12:46.937062 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:46.937020 2576 patch_prober.go:28] interesting pod/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Aug 04 21:12:46.937346 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:46.937317 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Aug 04 21:12:47.937818 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:47.937771 2576 patch_prober.go:28] interesting pod/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Aug 04 21:12:47.938362 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:47.937865 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Aug 04 21:12:48.940667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:48.940631 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:48.941068 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:48.941022 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:48.941769 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:48.941748 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:12:52.855453 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:52.855412 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:53.756108 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.756075 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd"] Aug 04 21:12:53.756416 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.756403 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" Aug 04 21:12:53.756461 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.756418 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" Aug 04 21:12:53.756461 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.756425 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" Aug 04 21:12:53.756461 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.756431 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" Aug 04 21:12:53.756552 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.756492 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" Aug 04 21:12:53.758497 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.758480 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" Aug 04 21:12:53.761468 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.761442 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 21:12:53.761468 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.761459 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86svs\"" Aug 04 21:12:53.767893 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.767868 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd"] Aug 04 21:12:53.832228 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.832189 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6629628-d57b-41bf-8d11-0dfccbfa3750-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd\" (UID: \"d6629628-d57b-41bf-8d11-0dfccbfa3750\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" Aug 04 21:12:53.933507 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.933474 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6629628-d57b-41bf-8d11-0dfccbfa3750-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd\" (UID: \"d6629628-d57b-41bf-8d11-0dfccbfa3750\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" Aug 04 21:12:53.934225 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:53.934191 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6629628-d57b-41bf-8d11-0dfccbfa3750-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd\" (UID: \"d6629628-d57b-41bf-8d11-0dfccbfa3750\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" Aug 04 21:12:54.070473 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:54.070396 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" Aug 04 21:12:54.199964 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:54.199939 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd"] Aug 04 21:12:54.202236 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:12:54.202210 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6629628_d57b_41bf_8d11_0dfccbfa3750.slice/crio-b39f337bde54caac32ce6e57c31719369f1173170bf8b45092b0969293d2660b WatchSource:0}: Error finding container b39f337bde54caac32ce6e57c31719369f1173170bf8b45092b0969293d2660b: Status 404 returned error can't find the container with id b39f337bde54caac32ce6e57c31719369f1173170bf8b45092b0969293d2660b Aug 04 21:12:54.745120 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:54.745018 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" event={"ID":"d6629628-d57b-41bf-8d11-0dfccbfa3750","Type":"ContainerStarted","Data":"97494d45ad3c4277aaf4c54297d0d74fdfb36c6b554c2185b1ba8e64dec19e3a"} Aug 04 21:12:54.745120 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:54.745087 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" event={"ID":"d6629628-d57b-41bf-8d11-0dfccbfa3750","Type":"ContainerStarted","Data":"b39f337bde54caac32ce6e57c31719369f1173170bf8b45092b0969293d2660b"} Aug 04 21:12:54.760339 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:54.760294 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" podStartSLOduration=1.76027885 podStartE2EDuration="1.76027885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:12:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:12:54.75913369 +0000 UTC m=+1908.906004244" watchObservedRunningTime="2026-08-04 21:12:54.76027885 +0000 UTC m=+1908.907149404" Aug 04 21:12:55.750761 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:55.750726 2576 generic.go:356] "Generic (PLEG): container finished" podID="d6629628-d57b-41bf-8d11-0dfccbfa3750" containerID="97494d45ad3c4277aaf4c54297d0d74fdfb36c6b554c2185b1ba8e64dec19e3a" exitCode=0 Aug 04 21:12:55.751169 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:55.750795 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" event={"ID":"d6629628-d57b-41bf-8d11-0dfccbfa3750","Type":"ContainerDied","Data":"97494d45ad3c4277aaf4c54297d0d74fdfb36c6b554c2185b1ba8e64dec19e3a"} Aug 04 21:12:56.886108 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:56.886081 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" Aug 04 21:12:56.957163 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:56.957129 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6629628-d57b-41bf-8d11-0dfccbfa3750-maas-api-service-ca\") pod \"d6629628-d57b-41bf-8d11-0dfccbfa3750\" (UID: \"d6629628-d57b-41bf-8d11-0dfccbfa3750\") " Aug 04 21:12:56.957498 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:56.957470 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6629628-d57b-41bf-8d11-0dfccbfa3750-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d6629628-d57b-41bf-8d11-0dfccbfa3750" (UID: "d6629628-d57b-41bf-8d11-0dfccbfa3750"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:12:57.058392 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:57.058309 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6629628-d57b-41bf-8d11-0dfccbfa3750-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:12:57.761504 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:57.761470 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" Aug 04 21:12:57.761680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:57.761501 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd" event={"ID":"d6629628-d57b-41bf-8d11-0dfccbfa3750","Type":"ContainerDied","Data":"b39f337bde54caac32ce6e57c31719369f1173170bf8b45092b0969293d2660b"} Aug 04 21:12:57.761680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:57.761550 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b39f337bde54caac32ce6e57c31719369f1173170bf8b45092b0969293d2660b" Aug 04 21:12:59.671919 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:59.671866 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:59.740144 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:59.740113 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:59.807752 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:59.807707 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:12:59.844159 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:59.844132 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd"] Aug 04 21:12:59.845577 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:59.845557 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-213b68bd-qxjnd"] Aug 04 21:12:59.872884 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:12:59.872848 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:00.439648 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:00.439613 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6629628-d57b-41bf-8d11-0dfccbfa3750" path="/var/lib/kubelet/pods/d6629628-d57b-41bf-8d11-0dfccbfa3750/volumes" Aug 04 21:13:00.981003 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:00.980972 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79"] Aug 04 21:13:00.981376 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:00.981186 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" containerID="cri-o://ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604" gracePeriod=30 Aug 04 21:13:00.992428 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:00.992400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:01.216151 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.216121 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:01.437635 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.437603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:01.717678 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.717589 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk"] Aug 04 21:13:01.718609 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.718570 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6629628-d57b-41bf-8d11-0dfccbfa3750" containerName="revoke-keys" Aug 04 21:13:01.718784 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.718771 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6629628-d57b-41bf-8d11-0dfccbfa3750" containerName="revoke-keys" Aug 04 21:13:01.719129 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.719113 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6629628-d57b-41bf-8d11-0dfccbfa3750" containerName="revoke-keys" Aug 04 21:13:01.719247 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.719235 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8e48f40-77e0-4e2f-a93f-129df46260e4" containerName="istio-proxy" Aug 04 21:13:01.722357 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.722333 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.727029 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.727009 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-043d8f1f-openshift-default-dockercfg-29vnz\"" Aug 04 21:13:01.736827 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.736803 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk"] Aug 04 21:13:01.739292 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.739267 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:01.802555 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.802519 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-certs\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.802792 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.802769 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-podinfo\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.802992 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.802963 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-credential-socket\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.803123 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.803003 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e216dc-f021-47fc-8e08-ac7e144e1a87-istiod-ca-cert\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.803123 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.803104 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-socket\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.803231 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.803138 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67v9v\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-kube-api-access-67v9v\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.803231 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.803187 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-envoy\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.803332 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.803233 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-data\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.803332 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.803288 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-token\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904482 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904443 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-socket\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904482 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904484 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67v9v\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-kube-api-access-67v9v\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904740 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904621 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-envoy\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904740 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904683 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-data\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904843 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-token\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904843 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904817 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-certs\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904985 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904853 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-podinfo\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904985 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904883 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-socket\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904985 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904935 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-credential-socket\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.904985 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.904970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e216dc-f021-47fc-8e08-ac7e144e1a87-istiod-ca-cert\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.905199 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.905081 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-data\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.905253 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.905200 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-certs\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.905301 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.905286 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-credential-socket\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.905632 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.905614 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e216dc-f021-47fc-8e08-ac7e144e1a87-istiod-ca-cert\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.907303 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.907281 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-envoy\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.907458 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.907442 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-podinfo\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.913560 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.913539 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-token\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.913654 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.913559 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67v9v\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-kube-api-access-67v9v\") pod \"e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:01.988761 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:01.988687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:02.033581 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:02.033560 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:02.172996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:02.172972 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk"] Aug 04 21:13:02.175546 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:13:02.175511 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e216dc_f021_47fc_8e08_ac7e144e1a87.slice/crio-b2108b82ec639a5f2c3227705f88c55c902a81bcc07a5217ef52e72b55176713 WatchSource:0}: Error finding container b2108b82ec639a5f2c3227705f88c55c902a81bcc07a5217ef52e72b55176713: Status 404 returned error can't find the container with id b2108b82ec639a5f2c3227705f88c55c902a81bcc07a5217ef52e72b55176713 Aug 04 21:13:02.177788 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:02.177756 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:13:02.177889 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:02.177821 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:13:02.177889 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:02.177862 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:13:02.786536 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:02.786498 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" event={"ID":"40e216dc-f021-47fc-8e08-ac7e144e1a87","Type":"ContainerStarted","Data":"eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12"} Aug 04 21:13:02.786714 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:02.786557 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" event={"ID":"40e216dc-f021-47fc-8e08-ac7e144e1a87","Type":"ContainerStarted","Data":"b2108b82ec639a5f2c3227705f88c55c902a81bcc07a5217ef52e72b55176713"} Aug 04 21:13:02.809688 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:02.809634 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" podStartSLOduration=1.80961324 podStartE2EDuration="1.80961324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:13:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:13:02.807818227 +0000 UTC m=+1916.954688784" watchObservedRunningTime="2026-08-04 21:13:02.80961324 +0000 UTC m=+1916.956483799" Aug 04 21:13:03.034215 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:03.034173 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:03.039242 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:03.039172 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:03.790996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:03.790967 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:03.791928 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:03.791881 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:03.943147 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:03.943112 2576 patch_prober.go:28] interesting pod/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 21:13:03.943314 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:03.943176 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 21:13:04.229427 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:04.229397 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:04.328842 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:04.328804 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:05.491109 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:05.491079 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:05.794254 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:05.794168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:05.973044 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:05.973017 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:06.072153 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.072069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:06.246060 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.246036 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:13:06.346940 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.346895 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-data\") pod \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " Aug 04 21:13:06.347102 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.346954 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-envoy\") pod \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " Aug 04 21:13:06.347102 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347022 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istiod-ca-cert\") pod \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " Aug 04 21:13:06.347102 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347040 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-podinfo\") pod \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " Aug 04 21:13:06.347102 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347059 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-certs\") pod \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " Aug 04 21:13:06.347102 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347101 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-token\") pod \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " Aug 04 21:13:06.347371 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347145 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-credential-socket\") pod \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " Aug 04 21:13:06.347371 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347173 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" (UID: "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:06.347371 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347180 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsq7c\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-kube-api-access-dsq7c\") pod \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " Aug 04 21:13:06.347526 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347390 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" (UID: "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:06.347526 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347456 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" (UID: "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:06.347526 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347471 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" (UID: "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:13:06.347526 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347500 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-socket\") pod \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\" (UID: \"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2\") " Aug 04 21:13:06.347880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347704 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" (UID: "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:06.347880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347834 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:06.347880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347848 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:06.347880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347857 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:06.347880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347865 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:06.347880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.347875 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:06.349384 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.349350 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" (UID: "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:06.349505 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.349396 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" (UID: "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:13:06.349843 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.349810 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" (UID: "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:13:06.349954 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.349839 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-kube-api-access-dsq7c" (OuterVolumeSpecName: "kube-api-access-dsq7c") pod "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" (UID: "f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2"). InnerVolumeSpecName "kube-api-access-dsq7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:13:06.449074 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.449040 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsq7c\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-kube-api-access-dsq7c\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:06.449074 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.449073 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:06.449260 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.449089 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:06.449260 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.449102 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:06.805818 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.805718 2576 generic.go:356] "Generic (PLEG): container finished" podID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerID="ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604" exitCode=0 Aug 04 21:13:06.805818 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.805782 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" Aug 04 21:13:06.805818 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.805806 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" event={"ID":"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2","Type":"ContainerDied","Data":"ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604"} Aug 04 21:13:06.806403 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.805865 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79" event={"ID":"f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2","Type":"ContainerDied","Data":"abe3121ea0b9ec762a90b4d7860f37d8a20df1d50d33d8640312acf80e21704a"} Aug 04 21:13:06.806403 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.805890 2576 scope.go:117] "RemoveContainer" containerID="ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604" Aug 04 21:13:06.814596 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.814577 2576 scope.go:117] "RemoveContainer" containerID="a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c" Aug 04 21:13:06.822298 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.822279 2576 scope.go:117] "RemoveContainer" containerID="ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604" Aug 04 21:13:06.822573 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:13:06.822546 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604\": container with ID starting with ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604 not found: ID does not exist" containerID="ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604" Aug 04 21:13:06.822669 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.822580 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604"} err="failed to get container status \"ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604\": rpc error: code = NotFound desc = could not find container \"ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604\": container with ID starting with ac486ef72088b48843009a86eec669356eb12590c6be791f8ce492efb7be8604 not found: ID does not exist" Aug 04 21:13:06.822669 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.822598 2576 scope.go:117] "RemoveContainer" containerID="a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c" Aug 04 21:13:06.822891 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:13:06.822876 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c\": container with ID starting with a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c not found: ID does not exist" containerID="a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c" Aug 04 21:13:06.822981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.822894 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c"} err="failed to get container status \"a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c\": rpc error: code = NotFound desc = could not find container \"a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c\": container with ID starting with a51da6ade0fc7f3432cafa885c127db3bba029afce2495779311701323a9d83c not found: ID does not exist" Aug 04 21:13:06.824493 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.824471 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79"] Aug 04 21:13:06.826300 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:06.826281 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-213b68bd-openshift-default-cc865cf8d-8gl79"] Aug 04 21:13:08.439980 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:08.439944 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" path="/var/lib/kubelet/pods/f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2/volumes" Aug 04 21:13:18.792499 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:18.792462 2576 patch_prober.go:28] interesting pod/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Aug 04 21:13:18.792954 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:18.792527 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Aug 04 21:13:18.874552 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:18.874517 2576 generic.go:356] "Generic (PLEG): container finished" podID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerID="eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12" exitCode=0 Aug 04 21:13:18.874709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:18.874600 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" event={"ID":"40e216dc-f021-47fc-8e08-ac7e144e1a87","Type":"ContainerDied","Data":"eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12"} Aug 04 21:13:18.875085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:18.875064 2576 scope.go:117] "RemoveContainer" containerID="eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12" Aug 04 21:13:18.876052 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:18.876027 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:13:18.876113 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:18.876088 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:13:18.876159 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:18.876120 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:13:19.881375 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:19.881335 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" event={"ID":"40e216dc-f021-47fc-8e08-ac7e144e1a87","Type":"ContainerStarted","Data":"5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867"} Aug 04 21:13:20.035459 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:20.034827 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:20.035459 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:20.035141 2576 patch_prober.go:28] interesting pod/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Aug 04 21:13:20.035459 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:20.035190 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Aug 04 21:13:21.034691 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:21.034657 2576 patch_prober.go:28] interesting pod/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Aug 04 21:13:21.035336 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:21.035288 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Aug 04 21:13:22.034638 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:22.034594 2576 patch_prober.go:28] interesting pod/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Aug 04 21:13:22.034990 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:22.034951 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Aug 04 21:13:23.034510 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:23.034460 2576 patch_prober.go:28] interesting pod/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Aug 04 21:13:23.034728 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:23.034536 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Aug 04 21:13:24.034781 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:24.034738 2576 patch_prober.go:28] interesting pod/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Aug 04 21:13:24.035400 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:24.035363 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Aug 04 21:13:25.037771 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:25.037740 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:25.038180 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:25.038141 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:25.038972 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:25.038950 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:29.029538 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.029503 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj"] Aug 04 21:13:29.029991 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.029840 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" Aug 04 21:13:29.029991 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.029853 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" Aug 04 21:13:29.029991 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.029862 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" Aug 04 21:13:29.029991 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.029869 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" Aug 04 21:13:29.029991 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.029936 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" Aug 04 21:13:29.029991 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.029948 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7ea91a6-3e5a-4fb4-aa22-6ac8642ca1b2" containerName="istio-proxy" Aug 04 21:13:29.032055 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.032039 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" Aug 04 21:13:29.034542 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.034521 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 21:13:29.034542 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.034533 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86svs\"" Aug 04 21:13:29.040708 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.040683 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj"] Aug 04 21:13:29.055275 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.055238 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03cd7977-d41f-4332-9a5d-4b31d3df0d77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj\" (UID: \"03cd7977-d41f-4332-9a5d-4b31d3df0d77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" Aug 04 21:13:29.156629 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.156585 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03cd7977-d41f-4332-9a5d-4b31d3df0d77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj\" (UID: \"03cd7977-d41f-4332-9a5d-4b31d3df0d77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" Aug 04 21:13:29.157230 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.157206 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03cd7977-d41f-4332-9a5d-4b31d3df0d77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj\" (UID: \"03cd7977-d41f-4332-9a5d-4b31d3df0d77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" Aug 04 21:13:29.343729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.343701 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" Aug 04 21:13:29.474893 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.474867 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj"] Aug 04 21:13:29.476897 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:13:29.476872 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03cd7977_d41f_4332_9a5d_4b31d3df0d77.slice/crio-322a6bfae7c3127df3c47c344ea4d3a53ec910d0f02241894d026e380cc1d489 WatchSource:0}: Error finding container 322a6bfae7c3127df3c47c344ea4d3a53ec910d0f02241894d026e380cc1d489: Status 404 returned error can't find the container with id 322a6bfae7c3127df3c47c344ea4d3a53ec910d0f02241894d026e380cc1d489 Aug 04 21:13:29.927016 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.926884 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" event={"ID":"03cd7977-d41f-4332-9a5d-4b31d3df0d77","Type":"ContainerStarted","Data":"116c755218866e6f979fae0cb35daa1dd25236c32d5f5b171e19288d234d8a4e"} Aug 04 21:13:29.927016 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.926988 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" event={"ID":"03cd7977-d41f-4332-9a5d-4b31d3df0d77","Type":"ContainerStarted","Data":"322a6bfae7c3127df3c47c344ea4d3a53ec910d0f02241894d026e380cc1d489"} Aug 04 21:13:29.943813 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:29.943754 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" podStartSLOduration=0.943737367 podStartE2EDuration="943.737367ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:13:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:13:29.942035906 +0000 UTC m=+1944.088906474" watchObservedRunningTime="2026-08-04 21:13:29.943737367 +0000 UTC m=+1944.090608131" Aug 04 21:13:30.933441 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:30.933403 2576 generic.go:356] "Generic (PLEG): container finished" podID="03cd7977-d41f-4332-9a5d-4b31d3df0d77" containerID="116c755218866e6f979fae0cb35daa1dd25236c32d5f5b171e19288d234d8a4e" exitCode=0 Aug 04 21:13:30.933864 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:30.933498 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" event={"ID":"03cd7977-d41f-4332-9a5d-4b31d3df0d77","Type":"ContainerDied","Data":"116c755218866e6f979fae0cb35daa1dd25236c32d5f5b171e19288d234d8a4e"} Aug 04 21:13:32.071027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:32.071000 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" Aug 04 21:13:32.184882 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:32.184849 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03cd7977-d41f-4332-9a5d-4b31d3df0d77-maas-api-service-ca\") pod \"03cd7977-d41f-4332-9a5d-4b31d3df0d77\" (UID: \"03cd7977-d41f-4332-9a5d-4b31d3df0d77\") " Aug 04 21:13:32.185251 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:32.185227 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03cd7977-d41f-4332-9a5d-4b31d3df0d77-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "03cd7977-d41f-4332-9a5d-4b31d3df0d77" (UID: "03cd7977-d41f-4332-9a5d-4b31d3df0d77"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:13:32.286125 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:32.286043 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03cd7977-d41f-4332-9a5d-4b31d3df0d77-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:32.944226 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:32.944192 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" Aug 04 21:13:32.944226 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:32.944212 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj" event={"ID":"03cd7977-d41f-4332-9a5d-4b31d3df0d77","Type":"ContainerDied","Data":"322a6bfae7c3127df3c47c344ea4d3a53ec910d0f02241894d026e380cc1d489"} Aug 04 21:13:32.944435 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:32.944262 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="322a6bfae7c3127df3c47c344ea4d3a53ec910d0f02241894d026e380cc1d489" Aug 04 21:13:36.102700 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.102664 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:36.203885 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.203848 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:36.278847 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.278805 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:36.320276 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.320243 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj"] Aug 04 21:13:36.322757 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.322729 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-043d8f1f-6cmvj"] Aug 04 21:13:36.342527 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.342484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:36.444685 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.444604 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03cd7977-d41f-4332-9a5d-4b31d3df0d77" path="/var/lib/kubelet/pods/03cd7977-d41f-4332-9a5d-4b31d3df0d77/volumes" Aug 04 21:13:36.473807 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.473770 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk"] Aug 04 21:13:36.474096 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.474064 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" containerID="cri-o://5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867" gracePeriod=30 Aug 04 21:13:36.475053 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.474614 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:36.699822 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:36.699725 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:37.159167 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:37.159135 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:37.232834 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:37.232799 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:37.294406 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:37.294367 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:40.040364 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:40.040325 2576 patch_prober.go:28] interesting pod/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 21:13:40.040893 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:40.040396 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 21:13:41.723066 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.723037 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:41.874803 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.874772 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-credential-socket\") pod \"40e216dc-f021-47fc-8e08-ac7e144e1a87\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " Aug 04 21:13:41.874996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.874808 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-podinfo\") pod \"40e216dc-f021-47fc-8e08-ac7e144e1a87\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " Aug 04 21:13:41.874996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.874848 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-envoy\") pod \"40e216dc-f021-47fc-8e08-ac7e144e1a87\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " Aug 04 21:13:41.874996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.874871 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e216dc-f021-47fc-8e08-ac7e144e1a87-istiod-ca-cert\") pod \"40e216dc-f021-47fc-8e08-ac7e144e1a87\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " Aug 04 21:13:41.874996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.874889 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67v9v\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-kube-api-access-67v9v\") pod \"40e216dc-f021-47fc-8e08-ac7e144e1a87\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " Aug 04 21:13:41.874996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.874951 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-token\") pod \"40e216dc-f021-47fc-8e08-ac7e144e1a87\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " Aug 04 21:13:41.874996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.874989 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-socket\") pod \"40e216dc-f021-47fc-8e08-ac7e144e1a87\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " Aug 04 21:13:41.875308 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875139 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40e216dc-f021-47fc-8e08-ac7e144e1a87" (UID: "40e216dc-f021-47fc-8e08-ac7e144e1a87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:41.875308 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875235 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40e216dc-f021-47fc-8e08-ac7e144e1a87" (UID: "40e216dc-f021-47fc-8e08-ac7e144e1a87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:41.875308 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875272 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-certs\") pod \"40e216dc-f021-47fc-8e08-ac7e144e1a87\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " Aug 04 21:13:41.875465 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875312 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-data\") pod \"40e216dc-f021-47fc-8e08-ac7e144e1a87\" (UID: \"40e216dc-f021-47fc-8e08-ac7e144e1a87\") " Aug 04 21:13:41.875465 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875311 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e216dc-f021-47fc-8e08-ac7e144e1a87-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40e216dc-f021-47fc-8e08-ac7e144e1a87" (UID: "40e216dc-f021-47fc-8e08-ac7e144e1a87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:13:41.875569 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875523 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40e216dc-f021-47fc-8e08-ac7e144e1a87" (UID: "40e216dc-f021-47fc-8e08-ac7e144e1a87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:41.875569 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875553 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-data" (OuterVolumeSpecName: "istio-data") pod "40e216dc-f021-47fc-8e08-ac7e144e1a87" (UID: "40e216dc-f021-47fc-8e08-ac7e144e1a87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:41.875844 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875820 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:41.875899 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875842 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:41.875899 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875857 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:41.875899 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875872 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e216dc-f021-47fc-8e08-ac7e144e1a87-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:41.875899 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.875884 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:41.877314 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.877288 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40e216dc-f021-47fc-8e08-ac7e144e1a87" (UID: "40e216dc-f021-47fc-8e08-ac7e144e1a87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:13:41.877421 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.877378 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40e216dc-f021-47fc-8e08-ac7e144e1a87" (UID: "40e216dc-f021-47fc-8e08-ac7e144e1a87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:13:41.877421 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.877395 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-token" (OuterVolumeSpecName: "istio-token") pod "40e216dc-f021-47fc-8e08-ac7e144e1a87" (UID: "40e216dc-f021-47fc-8e08-ac7e144e1a87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:13:41.877502 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.877449 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-kube-api-access-67v9v" (OuterVolumeSpecName: "kube-api-access-67v9v") pod "40e216dc-f021-47fc-8e08-ac7e144e1a87" (UID: "40e216dc-f021-47fc-8e08-ac7e144e1a87"). InnerVolumeSpecName "kube-api-access-67v9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:13:41.977296 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.977254 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:41.977296 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.977292 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:41.977296 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.977301 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e216dc-f021-47fc-8e08-ac7e144e1a87-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:41.977296 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.977309 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67v9v\" (UniqueName: \"kubernetes.io/projected/40e216dc-f021-47fc-8e08-ac7e144e1a87-kube-api-access-67v9v\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:13:41.987574 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.987545 2576 generic.go:356] "Generic (PLEG): container finished" podID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerID="5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867" exitCode=0 Aug 04 21:13:41.987694 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.987596 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" Aug 04 21:13:41.987694 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.987595 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" event={"ID":"40e216dc-f021-47fc-8e08-ac7e144e1a87","Type":"ContainerDied","Data":"5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867"} Aug 04 21:13:41.987774 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.987756 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk" event={"ID":"40e216dc-f021-47fc-8e08-ac7e144e1a87","Type":"ContainerDied","Data":"b2108b82ec639a5f2c3227705f88c55c902a81bcc07a5217ef52e72b55176713"} Aug 04 21:13:41.987822 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.987795 2576 scope.go:117] "RemoveContainer" containerID="5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867" Aug 04 21:13:41.997093 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:41.997075 2576 scope.go:117] "RemoveContainer" containerID="eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12" Aug 04 21:13:42.004751 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:42.004731 2576 scope.go:117] "RemoveContainer" containerID="5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867" Aug 04 21:13:42.005023 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:13:42.005000 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867\": container with ID starting with 5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867 not found: ID does not exist" containerID="5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867" Aug 04 21:13:42.005085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:42.005030 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867"} err="failed to get container status \"5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867\": rpc error: code = NotFound desc = could not find container \"5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867\": container with ID starting with 5d6c6671e2657e3eda54989c737cbed4f86877150c45efa98e06cb19dd15d867 not found: ID does not exist" Aug 04 21:13:42.005085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:42.005047 2576 scope.go:117] "RemoveContainer" containerID="eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12" Aug 04 21:13:42.005284 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:13:42.005268 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12\": container with ID starting with eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12 not found: ID does not exist" containerID="eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12" Aug 04 21:13:42.005326 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:42.005289 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12"} err="failed to get container status \"eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12\": rpc error: code = NotFound desc = could not find container \"eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12\": container with ID starting with eab053ab86546a0dbb645a1d25ef16c8fb6315a362731b77bfe788d0b6119d12 not found: ID does not exist" Aug 04 21:13:42.010705 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:42.010682 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk"] Aug 04 21:13:42.016709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:42.016688 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-043d8f1f-openshift-default-5b4f9ff768-vjjrk"] Aug 04 21:13:42.440279 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:42.440244 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" path="/var/lib/kubelet/pods/40e216dc-f021-47fc-8e08-ac7e144e1a87/volumes" Aug 04 21:13:49.033274 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:49.033240 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:56.098306 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:56.098248 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:56.199865 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:56.199831 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:56.300234 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:56.300198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:56.394074 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:56.393995 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:13:56.849183 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:13:56.849152 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:22.845050 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:22.845016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:26.210655 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:26.210619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:29.345585 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:29.345547 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:29.411918 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:29.411861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:29.485339 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:29.485302 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:29.586596 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:29.586565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:45.484413 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:45.484374 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:51.610179 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:51.610134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:51.711399 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:51.711352 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:51.811025 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:51.810988 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:14:51.911162 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:14:51.911079 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:00.144296 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144252 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72"] Aug 04 21:15:00.144709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144624 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" Aug 04 21:15:00.144709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144635 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" Aug 04 21:15:00.144709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144652 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" Aug 04 21:15:00.144709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144657 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" Aug 04 21:15:00.144709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144666 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03cd7977-d41f-4332-9a5d-4b31d3df0d77" containerName="revoke-keys" Aug 04 21:15:00.144709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144671 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="03cd7977-d41f-4332-9a5d-4b31d3df0d77" containerName="revoke-keys" Aug 04 21:15:00.144930 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144721 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="03cd7977-d41f-4332-9a5d-4b31d3df0d77" containerName="revoke-keys" Aug 04 21:15:00.144930 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144731 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" Aug 04 21:15:00.144930 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.144739 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e216dc-f021-47fc-8e08-ac7e144e1a87" containerName="istio-proxy" Aug 04 21:15:00.147733 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.147712 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72" Aug 04 21:15:00.150770 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.150749 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86svs\"" Aug 04 21:15:00.155787 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.155762 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72"] Aug 04 21:15:00.158521 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.158500 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72" Aug 04 21:15:00.292359 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.292334 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72"] Aug 04 21:15:00.295044 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:15:00.295013 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce242cf8_9969_4de3_8bfe_44e1439e619a.slice/crio-07f01d9a38e9083e82748efe6a9520139dc15b5e546483f66ef01e01396601d1 WatchSource:0}: Error finding container 07f01d9a38e9083e82748efe6a9520139dc15b5e546483f66ef01e01396601d1: Status 404 returned error can't find the container with id 07f01d9a38e9083e82748efe6a9520139dc15b5e546483f66ef01e01396601d1 Aug 04 21:15:00.346033 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:00.345993 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72" event={"ID":"ce242cf8-9969-4de3-8bfe-44e1439e619a","Type":"ContainerStarted","Data":"07f01d9a38e9083e82748efe6a9520139dc15b5e546483f66ef01e01396601d1"} Aug 04 21:15:01.352868 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:01.352828 2576 generic.go:356] "Generic (PLEG): container finished" podID="ce242cf8-9969-4de3-8bfe-44e1439e619a" containerID="b6c11fc582d5315f5139d7646a6b6d65176995a0671ab9f7593b68ba67f8970e" exitCode=0 Aug 04 21:15:01.353286 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:01.352926 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72" event={"ID":"ce242cf8-9969-4de3-8bfe-44e1439e619a","Type":"ContainerDied","Data":"b6c11fc582d5315f5139d7646a6b6d65176995a0671ab9f7593b68ba67f8970e"} Aug 04 21:15:02.439241 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:02.439210 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:02.487647 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:02.487625 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72" Aug 04 21:15:03.363132 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:03.363096 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72" event={"ID":"ce242cf8-9969-4de3-8bfe-44e1439e619a","Type":"ContainerDied","Data":"07f01d9a38e9083e82748efe6a9520139dc15b5e546483f66ef01e01396601d1"} Aug 04 21:15:03.363132 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:03.363130 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764635-jhk72" Aug 04 21:15:03.363340 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:03.363145 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07f01d9a38e9083e82748efe6a9520139dc15b5e546483f66ef01e01396601d1" Aug 04 21:15:07.942874 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:07.942841 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:13.378632 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:13.378594 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:19.342506 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:19.342469 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:19.449018 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:19.448980 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:19.543737 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:19.543697 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:19.650302 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:19.650199 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:20.317920 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:20.317862 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:20.419170 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:20.419122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:20.520503 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:20.520469 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:22.034476 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:22.034439 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:28.712254 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:28.712131 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:28.814311 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:28.814274 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:29.396424 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:29.396379 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:36.058791 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.058750 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:36.158608 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.158569 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:36.306667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.306625 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:36.590224 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.590184 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h"] Aug 04 21:15:36.590609 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.590590 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce242cf8-9969-4de3-8bfe-44e1439e619a" containerName="cleanup" Aug 04 21:15:36.590704 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.590613 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce242cf8-9969-4de3-8bfe-44e1439e619a" containerName="cleanup" Aug 04 21:15:36.590704 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.590700 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce242cf8-9969-4de3-8bfe-44e1439e619a" containerName="cleanup" Aug 04 21:15:36.593834 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.593803 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.596761 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.596741 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2fcd204a-openshift-default-dockercfg-k8vn5\"" Aug 04 21:15:36.608458 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.608433 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h"] Aug 04 21:15:36.618688 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.618661 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:36.768997 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.768963 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-socket\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.768997 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.768999 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-certs\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.769266 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.769046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-token\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.769266 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.769078 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-podinfo\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.769266 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.769096 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-envoy\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.769266 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.769124 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72xb9\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-kube-api-access-72xb9\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.769266 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.769226 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-data\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.769477 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.769271 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-credential-socket\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.769477 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.769308 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32867d2-4a23-4ded-b834-5a84a1f901cb-istiod-ca-cert\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.870615 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.870523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72xb9\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-kube-api-access-72xb9\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.870615 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.870585 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-data\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.870615 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.870610 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-credential-socket\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.870885 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.870630 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32867d2-4a23-4ded-b834-5a84a1f901cb-istiod-ca-cert\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.870885 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.870648 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-socket\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.870885 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.870670 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-certs\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.870885 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.870741 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-token\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.870885 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.870778 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-podinfo\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.870885 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.870803 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-envoy\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.871188 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.871096 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-credential-socket\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.871363 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.871338 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-data\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.871496 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.871403 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-socket\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.871496 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.871442 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-certs\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.871601 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.871529 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32867d2-4a23-4ded-b834-5a84a1f901cb-istiod-ca-cert\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.873290 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.873270 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-envoy\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.873617 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.873602 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-podinfo\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.879151 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.879127 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-token\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.879368 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.879348 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72xb9\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-kube-api-access-72xb9\") pod \"e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:36.904594 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:36.904569 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:37.005050 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:37.005013 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:37.043156 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:37.043131 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h"] Aug 04 21:15:37.045255 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:15:37.045227 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32867d2_4a23_4ded_b834_5a84a1f901cb.slice/crio-bc205df900a57da872094811f05181d9cc149e6f07fb4a94972fa104198de99b WatchSource:0}: Error finding container bc205df900a57da872094811f05181d9cc149e6f07fb4a94972fa104198de99b: Status 404 returned error can't find the container with id bc205df900a57da872094811f05181d9cc149e6f07fb4a94972fa104198de99b Aug 04 21:15:37.047474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:37.047441 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:15:37.047568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:37.047506 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:15:37.047568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:37.047532 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:15:37.524999 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:37.524959 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" event={"ID":"b32867d2-4a23-4ded-b834-5a84a1f901cb","Type":"ContainerStarted","Data":"26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e"} Aug 04 21:15:37.525371 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:37.525037 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" event={"ID":"b32867d2-4a23-4ded-b834-5a84a1f901cb","Type":"ContainerStarted","Data":"bc205df900a57da872094811f05181d9cc149e6f07fb4a94972fa104198de99b"} Aug 04 21:15:37.547987 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:37.547936 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" podStartSLOduration=1.547920586 podStartE2EDuration="1.547920586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:15:37.546609097 +0000 UTC m=+2071.693479643" watchObservedRunningTime="2026-08-04 21:15:37.547920586 +0000 UTC m=+2071.694791132" Aug 04 21:15:37.905458 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:37.905424 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:38.910379 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:38.910344 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:39.481509 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:39.481469 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:39.535433 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:39.535400 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:39.536372 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:39.536353 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:39.563726 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:39.563677 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:40.596980 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:40.596950 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:41.869184 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:41.869142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:15:53.608927 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:53.608875 2576 generic.go:356] "Generic (PLEG): container finished" podID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerID="26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e" exitCode=0 Aug 04 21:15:53.609404 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:53.608956 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" event={"ID":"b32867d2-4a23-4ded-b834-5a84a1f901cb","Type":"ContainerDied","Data":"26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e"} Aug 04 21:15:53.609404 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:53.609373 2576 scope.go:117] "RemoveContainer" containerID="26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e" Aug 04 21:15:53.610608 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:53.610577 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:15:53.610677 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:53.610660 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:15:53.610732 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:53.610718 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:15:54.536103 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:54.536055 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:54.615758 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:54.615723 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" event={"ID":"b32867d2-4a23-4ded-b834-5a84a1f901cb","Type":"ContainerStarted","Data":"df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876"} Aug 04 21:15:54.904985 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:54.904947 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:54.905946 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:54.905899 2576 patch_prober.go:28] interesting pod/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Aug 04 21:15:54.906003 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:54.905986 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Aug 04 21:15:55.905116 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:55.905081 2576 patch_prober.go:28] interesting pod/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Aug 04 21:15:55.905531 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:55.905138 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Aug 04 21:15:56.905757 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:56.905715 2576 patch_prober.go:28] interesting pod/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Aug 04 21:15:56.906223 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:56.905777 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Aug 04 21:15:57.905191 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:57.905151 2576 patch_prober.go:28] interesting pod/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Aug 04 21:15:57.905422 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:57.905213 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Aug 04 21:15:58.905770 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:58.905734 2576 patch_prober.go:28] interesting pod/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Aug 04 21:15:58.906205 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:58.905789 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Aug 04 21:15:59.909124 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:59.909094 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:59.909506 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:59.909359 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:15:59.909976 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:15:59.909958 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:16:04.492082 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:04.492036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:05.345574 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:05.345544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:05.416758 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:05.416717 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:05.615968 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:05.615861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:11.481371 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:11.481341 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:11.580481 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:11.580444 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:11.834897 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:11.834809 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg"] Aug 04 21:16:11.838225 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:11.838201 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" Aug 04 21:16:11.841155 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:11.841131 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 21:16:11.841266 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:11.841134 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86svs\"" Aug 04 21:16:11.847569 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:11.847543 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg"] Aug 04 21:16:11.986795 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:11.986758 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb793cc-82ee-4561-b428-6c8555caaf22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg\" (UID: \"7cb793cc-82ee-4561-b428-6c8555caaf22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" Aug 04 21:16:12.087830 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:12.087738 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb793cc-82ee-4561-b428-6c8555caaf22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg\" (UID: \"7cb793cc-82ee-4561-b428-6c8555caaf22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" Aug 04 21:16:12.088371 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:12.088349 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb793cc-82ee-4561-b428-6c8555caaf22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg\" (UID: \"7cb793cc-82ee-4561-b428-6c8555caaf22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" Aug 04 21:16:12.149383 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:12.149348 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" Aug 04 21:16:12.280710 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:12.280685 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg"] Aug 04 21:16:12.282503 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:16:12.282471 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cb793cc_82ee_4561_b428_6c8555caaf22.slice/crio-593a56fd3fbf3e6a15da083e6a50db2f31498077e8ea1e38bbdfd772db9c7951 WatchSource:0}: Error finding container 593a56fd3fbf3e6a15da083e6a50db2f31498077e8ea1e38bbdfd772db9c7951: Status 404 returned error can't find the container with id 593a56fd3fbf3e6a15da083e6a50db2f31498077e8ea1e38bbdfd772db9c7951 Aug 04 21:16:12.713383 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:12.713291 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" event={"ID":"7cb793cc-82ee-4561-b428-6c8555caaf22","Type":"ContainerStarted","Data":"925a66b1d44ba95ce80c7babdd173a48eb31fc9f36ca1cdbc3d404deb27a0445"} Aug 04 21:16:12.713383 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:12.713345 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" event={"ID":"7cb793cc-82ee-4561-b428-6c8555caaf22","Type":"ContainerStarted","Data":"593a56fd3fbf3e6a15da083e6a50db2f31498077e8ea1e38bbdfd772db9c7951"} Aug 04 21:16:12.730303 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:12.730248 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" podStartSLOduration=1.7302325029999999 podStartE2EDuration="1.730232503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:16:12.729208105 +0000 UTC m=+2106.876078661" watchObservedRunningTime="2026-08-04 21:16:12.730232503 +0000 UTC m=+2106.877103058" Aug 04 21:16:13.718990 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:13.718948 2576 generic.go:356] "Generic (PLEG): container finished" podID="7cb793cc-82ee-4561-b428-6c8555caaf22" containerID="925a66b1d44ba95ce80c7babdd173a48eb31fc9f36ca1cdbc3d404deb27a0445" exitCode=0 Aug 04 21:16:13.719437 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:13.719018 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" event={"ID":"7cb793cc-82ee-4561-b428-6c8555caaf22","Type":"ContainerDied","Data":"925a66b1d44ba95ce80c7babdd173a48eb31fc9f36ca1cdbc3d404deb27a0445"} Aug 04 21:16:14.855398 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:14.855369 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" Aug 04 21:16:15.016634 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:15.016546 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb793cc-82ee-4561-b428-6c8555caaf22-maas-api-service-ca\") pod \"7cb793cc-82ee-4561-b428-6c8555caaf22\" (UID: \"7cb793cc-82ee-4561-b428-6c8555caaf22\") " Aug 04 21:16:15.016880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:15.016858 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cb793cc-82ee-4561-b428-6c8555caaf22-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7cb793cc-82ee-4561-b428-6c8555caaf22" (UID: "7cb793cc-82ee-4561-b428-6c8555caaf22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:16:15.118038 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:15.118001 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb793cc-82ee-4561-b428-6c8555caaf22-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:15.734573 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:15.734547 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" Aug 04 21:16:15.734738 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:15.734546 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg" event={"ID":"7cb793cc-82ee-4561-b428-6c8555caaf22","Type":"ContainerDied","Data":"593a56fd3fbf3e6a15da083e6a50db2f31498077e8ea1e38bbdfd772db9c7951"} Aug 04 21:16:15.734738 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:15.734675 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="593a56fd3fbf3e6a15da083e6a50db2f31498077e8ea1e38bbdfd772db9c7951" Aug 04 21:16:22.294140 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:22.294100 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:22.373503 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:22.373470 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:22.463106 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:22.463073 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:22.469181 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:22.469152 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg"] Aug 04 21:16:22.471298 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:22.471274 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fcd204a-t9mhg"] Aug 04 21:16:22.578979 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:22.578832 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:22.829832 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:22.829717 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:24.440245 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:24.440210 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cb793cc-82ee-4561-b428-6c8555caaf22" path="/var/lib/kubelet/pods/7cb793cc-82ee-4561-b428-6c8555caaf22/volumes" Aug 04 21:16:28.995377 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:28.995343 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:29.092311 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.092274 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:29.251603 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.251513 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h"] Aug 04 21:16:29.251769 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.251719 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" containerID="cri-o://df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876" gracePeriod=30 Aug 04 21:16:29.279572 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.279533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:29.507375 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.507271 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:29.638250 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.638213 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:29.727245 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.727205 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:29.832147 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.832063 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:29.911969 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.911935 2576 patch_prober.go:28] interesting pod/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Aug 04 21:16:29.912137 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:29.911990 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 21:16:34.504711 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.504687 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:16:34.597365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597338 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-socket\") pod \"b32867d2-4a23-4ded-b834-5a84a1f901cb\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " Aug 04 21:16:34.597548 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597376 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-envoy\") pod \"b32867d2-4a23-4ded-b834-5a84a1f901cb\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " Aug 04 21:16:34.597548 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597393 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-data\") pod \"b32867d2-4a23-4ded-b834-5a84a1f901cb\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " Aug 04 21:16:34.597548 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597413 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-credential-socket\") pod \"b32867d2-4a23-4ded-b834-5a84a1f901cb\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " Aug 04 21:16:34.597548 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597462 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32867d2-4a23-4ded-b834-5a84a1f901cb-istiod-ca-cert\") pod \"b32867d2-4a23-4ded-b834-5a84a1f901cb\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " Aug 04 21:16:34.597548 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597482 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72xb9\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-kube-api-access-72xb9\") pod \"b32867d2-4a23-4ded-b834-5a84a1f901cb\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " Aug 04 21:16:34.597548 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597512 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-podinfo\") pod \"b32867d2-4a23-4ded-b834-5a84a1f901cb\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " Aug 04 21:16:34.597946 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597643 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-certs\") pod \"b32867d2-4a23-4ded-b834-5a84a1f901cb\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " Aug 04 21:16:34.597946 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597683 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-token\") pod \"b32867d2-4a23-4ded-b834-5a84a1f901cb\" (UID: \"b32867d2-4a23-4ded-b834-5a84a1f901cb\") " Aug 04 21:16:34.597946 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597707 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b32867d2-4a23-4ded-b834-5a84a1f901cb" (UID: "b32867d2-4a23-4ded-b834-5a84a1f901cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:16:34.597946 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.597753 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "b32867d2-4a23-4ded-b834-5a84a1f901cb" (UID: "b32867d2-4a23-4ded-b834-5a84a1f901cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:16:34.598326 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.598056 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b32867d2-4a23-4ded-b834-5a84a1f901cb" (UID: "b32867d2-4a23-4ded-b834-5a84a1f901cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:16:34.598326 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.598294 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b32867d2-4a23-4ded-b834-5a84a1f901cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b32867d2-4a23-4ded-b834-5a84a1f901cb" (UID: "b32867d2-4a23-4ded-b834-5a84a1f901cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:16:34.599183 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.598816 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b32867d2-4a23-4ded-b834-5a84a1f901cb" (UID: "b32867d2-4a23-4ded-b834-5a84a1f901cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:16:34.599183 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.598975 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:34.599183 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.598993 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:34.599183 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.599006 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:34.599183 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.599018 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32867d2-4a23-4ded-b834-5a84a1f901cb-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:34.599183 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.599031 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:34.600106 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.600075 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b32867d2-4a23-4ded-b834-5a84a1f901cb" (UID: "b32867d2-4a23-4ded-b834-5a84a1f901cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:16:34.600235 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.600200 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b32867d2-4a23-4ded-b834-5a84a1f901cb" (UID: "b32867d2-4a23-4ded-b834-5a84a1f901cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:16:34.600420 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.600400 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-kube-api-access-72xb9" (OuterVolumeSpecName: "kube-api-access-72xb9") pod "b32867d2-4a23-4ded-b834-5a84a1f901cb" (UID: "b32867d2-4a23-4ded-b834-5a84a1f901cb"). InnerVolumeSpecName "kube-api-access-72xb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:16:34.600549 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.600530 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "b32867d2-4a23-4ded-b834-5a84a1f901cb" (UID: "b32867d2-4a23-4ded-b834-5a84a1f901cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:16:34.699473 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.699439 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:34.699473 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.699466 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:34.699473 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.699474 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32867d2-4a23-4ded-b834-5a84a1f901cb-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:34.699473 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.699482 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72xb9\" (UniqueName: \"kubernetes.io/projected/b32867d2-4a23-4ded-b834-5a84a1f901cb-kube-api-access-72xb9\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:16:34.828152 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.828118 2576 generic.go:356] "Generic (PLEG): container finished" podID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerID="df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876" exitCode=0 Aug 04 21:16:34.828302 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.828188 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" event={"ID":"b32867d2-4a23-4ded-b834-5a84a1f901cb","Type":"ContainerDied","Data":"df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876"} Aug 04 21:16:34.828302 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.828205 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" Aug 04 21:16:34.828302 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.828267 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h" event={"ID":"b32867d2-4a23-4ded-b834-5a84a1f901cb","Type":"ContainerDied","Data":"bc205df900a57da872094811f05181d9cc149e6f07fb4a94972fa104198de99b"} Aug 04 21:16:34.828302 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.828290 2576 scope.go:117] "RemoveContainer" containerID="df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876" Aug 04 21:16:34.837543 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.837524 2576 scope.go:117] "RemoveContainer" containerID="26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e" Aug 04 21:16:34.845964 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.845944 2576 scope.go:117] "RemoveContainer" containerID="df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876" Aug 04 21:16:34.846242 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:16:34.846220 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876\": container with ID starting with df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876 not found: ID does not exist" containerID="df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876" Aug 04 21:16:34.846298 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.846251 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876"} err="failed to get container status \"df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876\": rpc error: code = NotFound desc = could not find container \"df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876\": container with ID starting with df1ad48c3d2dd0033976322d701f989183332a4267470bd8bd91326e13de9876 not found: ID does not exist" Aug 04 21:16:34.846298 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.846268 2576 scope.go:117] "RemoveContainer" containerID="26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e" Aug 04 21:16:34.846505 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:16:34.846488 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e\": container with ID starting with 26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e not found: ID does not exist" containerID="26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e" Aug 04 21:16:34.846546 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.846512 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e"} err="failed to get container status \"26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e\": rpc error: code = NotFound desc = could not find container \"26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e\": container with ID starting with 26a85b460ec34fd6b8d1587ad8b5377df37d91f2558e976dc5065a358f1a228e not found: ID does not exist" Aug 04 21:16:34.850125 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.850076 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h"] Aug 04 21:16:34.856193 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:34.856173 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2fcd204a-openshift-default-59ccc68b8d-4fk6h"] Aug 04 21:16:35.622996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:35.622967 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:35.728115 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:35.728074 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:35.823707 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:35.823668 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:35.984406 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:35.984315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:36.440579 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:36.440547 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" path="/var/lib/kubelet/pods/b32867d2-4a23-4ded-b834-5a84a1f901cb/volumes" Aug 04 21:16:42.158889 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:42.158847 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:42.258670 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:42.258632 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:16:45.531967 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:16:45.531931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:06.522871 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:06.522838 2576 scope.go:117] "RemoveContainer" containerID="5cf66c93528cd7534af20b0b57f41b8ae89aee7f8abb04cd681deb05b8628476" Aug 04 21:18:41.510391 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:41.510344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:45.313381 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.313341 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:45.604565 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.604528 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl"] Aug 04 21:18:45.604863 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.604851 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" Aug 04 21:18:45.604931 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.604865 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" Aug 04 21:18:45.604931 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.604874 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cb793cc-82ee-4561-b428-6c8555caaf22" containerName="revoke-keys" Aug 04 21:18:45.604931 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.604880 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cb793cc-82ee-4561-b428-6c8555caaf22" containerName="revoke-keys" Aug 04 21:18:45.605030 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.604980 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" Aug 04 21:18:45.605030 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.604990 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cb793cc-82ee-4561-b428-6c8555caaf22" containerName="revoke-keys" Aug 04 21:18:45.605030 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.604997 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" Aug 04 21:18:45.607803 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.607786 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.610374 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.610343 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d1f5ec52-openshift-default-dockercfg-z9tvj\"" Aug 04 21:18:45.623989 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.623952 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl"] Aug 04 21:18:45.634339 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.634303 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:45.720737 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.720701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-envoy\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.720737 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.720734 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-data\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.720971 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.720767 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-socket\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.720971 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.720886 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-certs\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.720971 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.720937 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562f3875-d773-4331-92e4-090d8331c925-istio-podinfo\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.721074 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.720984 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-credential-socket\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.721074 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.721016 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562f3875-d773-4331-92e4-090d8331c925-istiod-ca-cert\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.721074 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.721054 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-istio-token\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.721166 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.721102 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z59kf\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-kube-api-access-z59kf\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822312 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822276 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-envoy\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822312 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822319 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-data\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-socket\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822412 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-certs\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822442 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562f3875-d773-4331-92e4-090d8331c925-istio-podinfo\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822472 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-credential-socket\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822513 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562f3875-d773-4331-92e4-090d8331c925-istiod-ca-cert\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822534 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-istio-token\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822871 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822571 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z59kf\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-kube-api-access-z59kf\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.822871 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.822810 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-data\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.823065 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.823044 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-socket\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.823118 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.823062 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-certs\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.823299 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.823279 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562f3875-d773-4331-92e4-090d8331c925-istiod-ca-cert\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.823345 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.823315 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-credential-socket\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.824787 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.824761 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-envoy\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.825191 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.825164 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562f3875-d773-4331-92e4-090d8331c925-istio-podinfo\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.831012 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.830983 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-istio-token\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.831662 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.831637 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z59kf\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-kube-api-access-z59kf\") pod \"e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:45.844477 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.844453 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:45.919945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:45.919815 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:46.066268 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:46.066211 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl"] Aug 04 21:18:46.069341 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:18:46.069292 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562f3875_d773_4331_92e4_090d8331c925.slice/crio-1da1532b3b7c4f5cd39aa9aafdbf99422a72e355b5c8541df009685465e5bce0 WatchSource:0}: Error finding container 1da1532b3b7c4f5cd39aa9aafdbf99422a72e355b5c8541df009685465e5bce0: Status 404 returned error can't find the container with id 1da1532b3b7c4f5cd39aa9aafdbf99422a72e355b5c8541df009685465e5bce0 Aug 04 21:18:46.072781 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:46.072740 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:18:46.072884 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:46.072830 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:18:46.072884 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:46.072879 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:18:46.451655 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:46.451612 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" event={"ID":"562f3875-d773-4331-92e4-090d8331c925","Type":"ContainerStarted","Data":"f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49"} Aug 04 21:18:46.452081 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:46.451676 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" event={"ID":"562f3875-d773-4331-92e4-090d8331c925","Type":"ContainerStarted","Data":"1da1532b3b7c4f5cd39aa9aafdbf99422a72e355b5c8541df009685465e5bce0"} Aug 04 21:18:46.475056 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:46.475006 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" podStartSLOduration=1.4749913829999999 podStartE2EDuration="1.474991383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:18:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:18:46.472600587 +0000 UTC m=+2260.619471141" watchObservedRunningTime="2026-08-04 21:18:46.474991383 +0000 UTC m=+2260.621861938" Aug 04 21:18:46.920738 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:46.920701 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:47.925209 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:47.925167 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:48.427269 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:48.427225 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:48.460190 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:48.460159 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:48.461289 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:48.461268 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:18:48.524570 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:48.524531 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:49.606130 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:49.606094 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:49.903210 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:49.903098 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:50.180528 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:50.180433 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:18:50.305138 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:18:50.305099 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:19:02.528748 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:02.528710 2576 generic.go:356] "Generic (PLEG): container finished" podID="562f3875-d773-4331-92e4-090d8331c925" containerID="f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49" exitCode=0 Aug 04 21:19:02.529231 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:02.528766 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" event={"ID":"562f3875-d773-4331-92e4-090d8331c925","Type":"ContainerDied","Data":"f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49"} Aug 04 21:19:02.529231 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:02.529148 2576 scope.go:117] "RemoveContainer" containerID="f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49" Aug 04 21:19:02.530154 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:02.530126 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:19:02.530218 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:02.530194 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:19:02.530255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:02.530224 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:19:03.460211 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:03.460166 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:19:03.537888 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:03.537844 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" event={"ID":"562f3875-d773-4331-92e4-090d8331c925","Type":"ContainerStarted","Data":"d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c"} Aug 04 21:19:03.920636 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:03.920594 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:19:03.921767 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:03.921747 2576 patch_prober.go:28] interesting pod/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Aug 04 21:19:03.921848 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:03.921795 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Aug 04 21:19:04.920309 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:04.920265 2576 patch_prober.go:28] interesting pod/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Aug 04 21:19:04.920710 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:04.920331 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Aug 04 21:19:05.920755 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:05.920719 2576 patch_prober.go:28] interesting pod/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Aug 04 21:19:05.921186 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:05.920791 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Aug 04 21:19:06.542983 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:06.542835 2576 scope.go:117] "RemoveContainer" containerID="97494d45ad3c4277aaf4c54297d0d74fdfb36c6b554c2185b1ba8e64dec19e3a" Aug 04 21:19:06.553600 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:06.553576 2576 scope.go:117] "RemoveContainer" containerID="a987e402f74edf5cf61e94e48c87faa5999c0e31c8171d6016460f5647c7ee28" Aug 04 21:19:06.920795 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:06.920763 2576 patch_prober.go:28] interesting pod/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Aug 04 21:19:06.921220 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:06.920815 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Aug 04 21:19:07.920363 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:07.920324 2576 patch_prober.go:28] interesting pod/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Aug 04 21:19:07.920541 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:07.920385 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Aug 04 21:19:08.949675 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:08.949638 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:19:08.950159 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:08.950021 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:19:08.950793 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:08.950771 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:19:13.406879 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:13.406844 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:19:20.981391 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:20.981357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:19:47.824587 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:47.824553 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5"] Aug 04 21:19:47.825085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:47.824924 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" Aug 04 21:19:47.825085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:47.824937 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b32867d2-4a23-4ded-b834-5a84a1f901cb" containerName="istio-proxy" Aug 04 21:19:47.827751 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:47.827735 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" Aug 04 21:19:47.830261 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:47.830240 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86svs\"" Aug 04 21:19:47.830391 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:47.830242 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 21:19:47.836419 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:47.836397 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5"] Aug 04 21:19:47.969826 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:47.969789 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9761c3-1e55-4cd7-a0c7-bec5443b36d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5\" (UID: \"8c9761c3-1e55-4cd7-a0c7-bec5443b36d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" Aug 04 21:19:48.070374 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:48.070341 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9761c3-1e55-4cd7-a0c7-bec5443b36d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5\" (UID: \"8c9761c3-1e55-4cd7-a0c7-bec5443b36d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" Aug 04 21:19:48.070953 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:48.070900 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9761c3-1e55-4cd7-a0c7-bec5443b36d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5\" (UID: \"8c9761c3-1e55-4cd7-a0c7-bec5443b36d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" Aug 04 21:19:48.138712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:48.138686 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" Aug 04 21:19:48.269161 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:48.269136 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5"] Aug 04 21:19:48.271310 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:19:48.271281 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9761c3_1e55_4cd7_a0c7_bec5443b36d2.slice/crio-b74b143c773daa9edf4ef9b9236fdc431782a574f156138cd13cfb45d1b55a31 WatchSource:0}: Error finding container b74b143c773daa9edf4ef9b9236fdc431782a574f156138cd13cfb45d1b55a31: Status 404 returned error can't find the container with id b74b143c773daa9edf4ef9b9236fdc431782a574f156138cd13cfb45d1b55a31 Aug 04 21:19:48.752494 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:48.752397 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" event={"ID":"8c9761c3-1e55-4cd7-a0c7-bec5443b36d2","Type":"ContainerStarted","Data":"60db6bfa58a9472d16fd16eb77a36d9dfb7798eac734b29a274b4783fe6bed1f"} Aug 04 21:19:48.752494 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:48.752455 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" event={"ID":"8c9761c3-1e55-4cd7-a0c7-bec5443b36d2","Type":"ContainerStarted","Data":"b74b143c773daa9edf4ef9b9236fdc431782a574f156138cd13cfb45d1b55a31"} Aug 04 21:19:48.767697 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:48.767647 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" podStartSLOduration=1.767633786 podStartE2EDuration="1.767633786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:19:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:19:48.766219852 +0000 UTC m=+2322.913090410" watchObservedRunningTime="2026-08-04 21:19:48.767633786 +0000 UTC m=+2322.914504381" Aug 04 21:19:49.761184 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:49.761147 2576 generic.go:356] "Generic (PLEG): container finished" podID="8c9761c3-1e55-4cd7-a0c7-bec5443b36d2" containerID="60db6bfa58a9472d16fd16eb77a36d9dfb7798eac734b29a274b4783fe6bed1f" exitCode=0 Aug 04 21:19:49.761596 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:49.761236 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" event={"ID":"8c9761c3-1e55-4cd7-a0c7-bec5443b36d2","Type":"ContainerDied","Data":"60db6bfa58a9472d16fd16eb77a36d9dfb7798eac734b29a274b4783fe6bed1f"} Aug 04 21:19:50.892360 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:50.892328 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" Aug 04 21:19:50.892649 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:50.892484 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9761c3-1e55-4cd7-a0c7-bec5443b36d2-maas-api-service-ca\") pod \"8c9761c3-1e55-4cd7-a0c7-bec5443b36d2\" (UID: \"8c9761c3-1e55-4cd7-a0c7-bec5443b36d2\") " Aug 04 21:19:50.892819 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:50.892798 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c9761c3-1e55-4cd7-a0c7-bec5443b36d2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c9761c3-1e55-4cd7-a0c7-bec5443b36d2" (UID: "8c9761c3-1e55-4cd7-a0c7-bec5443b36d2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:19:50.993836 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:50.993800 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9761c3-1e55-4cd7-a0c7-bec5443b36d2-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:19:51.771206 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:51.771173 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" Aug 04 21:19:51.771380 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:51.771222 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5" event={"ID":"8c9761c3-1e55-4cd7-a0c7-bec5443b36d2","Type":"ContainerDied","Data":"b74b143c773daa9edf4ef9b9236fdc431782a574f156138cd13cfb45d1b55a31"} Aug 04 21:19:51.771380 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:19:51.771269 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b74b143c773daa9edf4ef9b9236fdc431782a574f156138cd13cfb45d1b55a31" Aug 04 21:20:03.276422 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:03.276338 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:03.341505 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:03.341460 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:03.444018 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:03.443978 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5"] Aug 04 21:20:03.446624 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:03.446581 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d1f5ec52-jj7q5"] Aug 04 21:20:03.504047 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:03.504012 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:03.568065 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:03.567965 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:04.439862 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:04.439829 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9761c3-1e55-4cd7-a0c7-bec5443b36d2" path="/var/lib/kubelet/pods/8c9761c3-1e55-4cd7-a0c7-bec5443b36d2/volumes" Aug 04 21:20:06.575102 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:06.575064 2576 scope.go:117] "RemoveContainer" containerID="116c755218866e6f979fae0cb35daa1dd25236c32d5f5b171e19288d234d8a4e" Aug 04 21:20:06.690784 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:06.690744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:11.755180 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:11.755131 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:48.289658 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:48.289622 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:49.499323 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:49.499285 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:54.814806 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:54.814765 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:54.915502 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:54.915466 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:55.069982 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:55.069865 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl"] Aug 04 21:20:55.070149 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:55.070127 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" containerID="cri-o://d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c" gracePeriod=30 Aug 04 21:20:55.084303 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:55.084268 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:55.318161 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:55.318126 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:20:55.975516 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:20:55.975483 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:00.319716 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.319689 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:21:00.399088 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399045 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-socket\") pod \"562f3875-d773-4331-92e4-090d8331c925\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " Aug 04 21:21:00.399088 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399090 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-istio-token\") pod \"562f3875-d773-4331-92e4-090d8331c925\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " Aug 04 21:21:00.399365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399117 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-data\") pod \"562f3875-d773-4331-92e4-090d8331c925\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " Aug 04 21:21:00.399365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399141 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-envoy\") pod \"562f3875-d773-4331-92e4-090d8331c925\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " Aug 04 21:21:00.399365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399158 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-certs\") pod \"562f3875-d773-4331-92e4-090d8331c925\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " Aug 04 21:21:00.399365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399174 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562f3875-d773-4331-92e4-090d8331c925-istio-podinfo\") pod \"562f3875-d773-4331-92e4-090d8331c925\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " Aug 04 21:21:00.399365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399208 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-credential-socket\") pod \"562f3875-d773-4331-92e4-090d8331c925\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " Aug 04 21:21:00.399365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399250 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562f3875-d773-4331-92e4-090d8331c925-istiod-ca-cert\") pod \"562f3875-d773-4331-92e4-090d8331c925\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " Aug 04 21:21:00.399365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399298 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z59kf\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-kube-api-access-z59kf\") pod \"562f3875-d773-4331-92e4-090d8331c925\" (UID: \"562f3875-d773-4331-92e4-090d8331c925\") " Aug 04 21:21:00.399712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399426 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-data" (OuterVolumeSpecName: "istio-data") pod "562f3875-d773-4331-92e4-090d8331c925" (UID: "562f3875-d773-4331-92e4-090d8331c925"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:21:00.399712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399450 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "562f3875-d773-4331-92e4-090d8331c925" (UID: "562f3875-d773-4331-92e4-090d8331c925"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:21:00.399712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399637 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:21:00.399712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399617 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "562f3875-d773-4331-92e4-090d8331c925" (UID: "562f3875-d773-4331-92e4-090d8331c925"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:21:00.399712 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399657 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:21:00.400183 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399780 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562f3875-d773-4331-92e4-090d8331c925-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "562f3875-d773-4331-92e4-090d8331c925" (UID: "562f3875-d773-4331-92e4-090d8331c925"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:21:00.400183 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.399780 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "562f3875-d773-4331-92e4-090d8331c925" (UID: "562f3875-d773-4331-92e4-090d8331c925"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:21:00.401588 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.401563 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-istio-token" (OuterVolumeSpecName: "istio-token") pod "562f3875-d773-4331-92e4-090d8331c925" (UID: "562f3875-d773-4331-92e4-090d8331c925"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:21:00.401680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.401583 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "562f3875-d773-4331-92e4-090d8331c925" (UID: "562f3875-d773-4331-92e4-090d8331c925"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:21:00.401680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.401647 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/562f3875-d773-4331-92e4-090d8331c925-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "562f3875-d773-4331-92e4-090d8331c925" (UID: "562f3875-d773-4331-92e4-090d8331c925"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:21:00.401750 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.401708 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-kube-api-access-z59kf" (OuterVolumeSpecName: "kube-api-access-z59kf") pod "562f3875-d773-4331-92e4-090d8331c925" (UID: "562f3875-d773-4331-92e4-090d8331c925"). InnerVolumeSpecName "kube-api-access-z59kf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:21:00.501047 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.501008 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:21:00.501047 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.501040 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:21:00.501047 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.501049 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:21:00.501047 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.501057 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562f3875-d773-4331-92e4-090d8331c925-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:21:00.501315 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.501065 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562f3875-d773-4331-92e4-090d8331c925-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:21:00.501315 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.501073 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562f3875-d773-4331-92e4-090d8331c925-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:21:00.501315 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:00.501081 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z59kf\" (UniqueName: \"kubernetes.io/projected/562f3875-d773-4331-92e4-090d8331c925-kube-api-access-z59kf\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:21:01.086711 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.086675 2576 generic.go:356] "Generic (PLEG): container finished" podID="562f3875-d773-4331-92e4-090d8331c925" containerID="d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c" exitCode=0 Aug 04 21:21:01.086888 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.086741 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" Aug 04 21:21:01.086888 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.086757 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" event={"ID":"562f3875-d773-4331-92e4-090d8331c925","Type":"ContainerDied","Data":"d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c"} Aug 04 21:21:01.086888 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.086814 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl" event={"ID":"562f3875-d773-4331-92e4-090d8331c925","Type":"ContainerDied","Data":"1da1532b3b7c4f5cd39aa9aafdbf99422a72e355b5c8541df009685465e5bce0"} Aug 04 21:21:01.086888 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.086838 2576 scope.go:117] "RemoveContainer" containerID="d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c" Aug 04 21:21:01.095397 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.095380 2576 scope.go:117] "RemoveContainer" containerID="f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49" Aug 04 21:21:01.103432 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.103415 2576 scope.go:117] "RemoveContainer" containerID="d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c" Aug 04 21:21:01.103710 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:21:01.103690 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c\": container with ID starting with d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c not found: ID does not exist" containerID="d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c" Aug 04 21:21:01.103762 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.103720 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c"} err="failed to get container status \"d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c\": rpc error: code = NotFound desc = could not find container \"d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c\": container with ID starting with d176ff44f5256152c02c85d50b8eac95a69a5850a7ca360cbe5c802e83fe6e3c not found: ID does not exist" Aug 04 21:21:01.103762 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.103737 2576 scope.go:117] "RemoveContainer" containerID="f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49" Aug 04 21:21:01.103986 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:21:01.103967 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49\": container with ID starting with f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49 not found: ID does not exist" containerID="f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49" Aug 04 21:21:01.104048 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.103995 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49"} err="failed to get container status \"f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49\": rpc error: code = NotFound desc = could not find container \"f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49\": container with ID starting with f79657c64beb69bed593fe1582f6fd933731d250138048e43976aacfb7461b49 not found: ID does not exist" Aug 04 21:21:01.108570 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.108549 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl"] Aug 04 21:21:01.112345 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:01.112325 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d1f5ec52-openshift-default-6cb7cb79c6-6w8wl"] Aug 04 21:21:02.440336 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:02.440305 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="562f3875-d773-4331-92e4-090d8331c925" path="/var/lib/kubelet/pods/562f3875-d773-4331-92e4-090d8331c925/volumes" Aug 04 21:21:03.232638 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:03.232601 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:06.615273 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:06.615237 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:06.715458 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:06.715419 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:06.814846 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:06.814804 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:07.437759 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:07.437721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:07.542698 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:07.542668 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:07.637719 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:07.637687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:08.918710 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:08.918673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:15.071726 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:15.071688 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:15.172320 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:15.172284 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:15.749604 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:15.749568 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:21.728132 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:21.728090 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:21.838550 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:21.838514 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:21.929251 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:21.929222 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:32.433196 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:32.433100 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:32.670468 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:32.670432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:53.629805 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:53.629764 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:21:59.980094 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:21:59.980060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:00.043467 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.043436 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:00.337537 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.337451 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:00.627079 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627048 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz"] Aug 04 21:22:00.627415 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627401 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" Aug 04 21:22:00.627459 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627417 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" Aug 04 21:22:00.627459 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627428 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9761c3-1e55-4cd7-a0c7-bec5443b36d2" containerName="revoke-keys" Aug 04 21:22:00.627459 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627434 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9761c3-1e55-4cd7-a0c7-bec5443b36d2" containerName="revoke-keys" Aug 04 21:22:00.627459 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627448 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" Aug 04 21:22:00.627459 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627454 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" Aug 04 21:22:00.627607 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627513 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" Aug 04 21:22:00.627607 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627524 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="562f3875-d773-4331-92e4-090d8331c925" containerName="istio-proxy" Aug 04 21:22:00.627607 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.627529 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9761c3-1e55-4cd7-a0c7-bec5443b36d2" containerName="revoke-keys" Aug 04 21:22:00.630442 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.630426 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.633029 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.633011 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-19ad13-openshift-default-dockercfg-4nfcg\"" Aug 04 21:22:00.642318 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.642291 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz"] Aug 04 21:22:00.654717 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.654687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:00.718680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.718635 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-socket\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.718863 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.718684 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9gns\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-kube-api-access-q9gns\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.718863 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.718718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94689d38-f839-409f-89e2-534f0eac89cd-istiod-ca-cert\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.718863 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.718738 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-data\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.719041 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.718853 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-istio-token\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.719041 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.718950 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-certs\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.719041 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.718980 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94689d38-f839-409f-89e2-534f0eac89cd-istio-podinfo\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.719041 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.719015 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-credential-socket\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.719194 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.719053 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-envoy\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.819785 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.819746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-certs\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.819785 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.819783 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94689d38-f839-409f-89e2-534f0eac89cd-istio-podinfo\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.819805 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-credential-socket\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.819827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-envoy\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.819894 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-socket\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.819952 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9gns\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-kube-api-access-q9gns\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.819976 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94689d38-f839-409f-89e2-534f0eac89cd-istiod-ca-cert\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.820003 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-data\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820085 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.820037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-istio-token\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820449 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.820224 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-certs\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820449 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.820274 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-credential-socket\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820449 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.820371 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-data\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820449 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.820385 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-socket\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.820774 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.820749 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94689d38-f839-409f-89e2-534f0eac89cd-istiod-ca-cert\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.822552 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.822526 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-envoy\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.822649 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.822571 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94689d38-f839-409f-89e2-534f0eac89cd-istio-podinfo\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.830862 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.830837 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-istio-token\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.831011 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.830993 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9gns\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-kube-api-access-q9gns\") pod \"e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:00.881821 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.881751 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:00.940482 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:00.940448 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:01.075431 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:01.075405 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz"] Aug 04 21:22:01.077648 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:22:01.077620 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94689d38_f839_409f_89e2_534f0eac89cd.slice/crio-2432932d0871d52cd75f576e1f242a25cc1b5e93266fb650b3be90227cb962cc WatchSource:0}: Error finding container 2432932d0871d52cd75f576e1f242a25cc1b5e93266fb650b3be90227cb962cc: Status 404 returned error can't find the container with id 2432932d0871d52cd75f576e1f242a25cc1b5e93266fb650b3be90227cb962cc Aug 04 21:22:01.079682 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:01.079653 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:01.079747 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:01.079715 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:01.079747 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:01.079744 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:01.374083 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:01.374044 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" event={"ID":"94689d38-f839-409f-89e2-534f0eac89cd","Type":"ContainerStarted","Data":"c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5"} Aug 04 21:22:01.374255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:01.374116 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" event={"ID":"94689d38-f839-409f-89e2-534f0eac89cd","Type":"ContainerStarted","Data":"2432932d0871d52cd75f576e1f242a25cc1b5e93266fb650b3be90227cb962cc"} Aug 04 21:22:01.395461 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:01.395355 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" podStartSLOduration=1.3953340650000001 podStartE2EDuration="1.395334065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:22:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:22:01.393444614 +0000 UTC m=+2455.540315179" watchObservedRunningTime="2026-08-04 21:22:01.395334065 +0000 UTC m=+2455.542204621" Aug 04 21:22:01.940559 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:01.940524 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:02.945829 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:02.945781 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:03.314164 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:03.314054 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:03.383223 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:03.383188 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:03.384133 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:03.384117 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:03.414747 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:03.414713 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:04.447577 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:04.447546 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:04.692804 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:04.692770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:05.043474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:05.043435 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:05.144721 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:05.144673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:17.447463 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:17.447378 2576 generic.go:356] "Generic (PLEG): container finished" podID="94689d38-f839-409f-89e2-534f0eac89cd" containerID="c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5" exitCode=0 Aug 04 21:22:17.447463 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:17.447453 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" event={"ID":"94689d38-f839-409f-89e2-534f0eac89cd","Type":"ContainerDied","Data":"c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5"} Aug 04 21:22:17.447897 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:17.447859 2576 scope.go:117] "RemoveContainer" containerID="c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5" Aug 04 21:22:17.448874 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:17.448847 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:17.448966 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:17.448949 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:17.449018 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:17.449002 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:18.383795 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:18.383755 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:18.456465 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:18.456415 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" event={"ID":"94689d38-f839-409f-89e2-534f0eac89cd","Type":"ContainerStarted","Data":"083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf"} Aug 04 21:22:18.942242 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:18.942197 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:18.942464 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:18.942324 2576 patch_prober.go:28] interesting pod/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Aug 04 21:22:18.942464 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:18.942385 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Aug 04 21:22:19.941766 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:19.941723 2576 patch_prober.go:28] interesting pod/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Aug 04 21:22:19.942548 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:19.942508 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Aug 04 21:22:20.941321 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:20.941267 2576 patch_prober.go:28] interesting pod/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Aug 04 21:22:20.941532 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:20.941360 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Aug 04 21:22:21.941136 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:21.941098 2576 patch_prober.go:28] interesting pod/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Aug 04 21:22:21.941700 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:21.941671 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Aug 04 21:22:22.941705 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:22.941665 2576 patch_prober.go:28] interesting pod/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Aug 04 21:22:22.942339 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:22.942307 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Aug 04 21:22:23.944933 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:23.944880 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:23.945351 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:23.945294 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:23.946069 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:23.946046 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:22:28.342190 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.342148 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:28.674274 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.674171 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:28.955974 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.955874 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl"] Aug 04 21:22:28.959496 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.959475 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.962159 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.962126 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4c3747-openshift-default-dockercfg-wpz6b\"" Aug 04 21:22:28.972444 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.972416 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl"] Aug 04 21:22:28.980056 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.980030 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-istio-token\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.980176 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.980083 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-credential-socket\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.980176 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.980113 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-socket\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.980176 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.980135 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f07096-4393-463a-b854-1ea0c19e2df0-istiod-ca-cert\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.980176 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.980167 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-certs\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.980341 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.980205 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-data\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.980341 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.980221 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fldvk\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-kube-api-access-fldvk\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.980341 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.980269 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-envoy\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.980341 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.980297 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f07096-4393-463a-b854-1ea0c19e2df0-istio-podinfo\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:28.984460 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:28.984432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:29.081122 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081082 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f07096-4393-463a-b854-1ea0c19e2df0-istio-podinfo\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.081328 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081143 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-istio-token\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.081328 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081191 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-credential-socket\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.081328 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-socket\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.081328 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f07096-4393-463a-b854-1ea0c19e2df0-istiod-ca-cert\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.081328 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-certs\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.081601 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081337 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-data\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.081601 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081362 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fldvk\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-kube-api-access-fldvk\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.081601 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081406 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-envoy\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.081840 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.081799 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-socket\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.082112 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.082085 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-data\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.082367 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.082307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-certs\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.082367 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.082321 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-credential-socket\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.082720 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.082696 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f07096-4393-463a-b854-1ea0c19e2df0-istiod-ca-cert\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.084126 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.084104 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-envoy\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.084311 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.084292 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f07096-4393-463a-b854-1ea0c19e2df0-istio-podinfo\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.090215 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.090192 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-istio-token\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.090541 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.090518 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fldvk\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-kube-api-access-fldvk\") pod \"e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.169706 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.169670 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:29.269521 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.269418 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:29.412267 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.412234 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl"] Aug 04 21:22:29.415549 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:22:29.415518 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f07096_4393_463a_b854_1ea0c19e2df0.slice/crio-92298121da6c03198c569330fb1e5b586f4fb49058faf85c232667b3813aa4e4 WatchSource:0}: Error finding container 92298121da6c03198c569330fb1e5b586f4fb49058faf85c232667b3813aa4e4: Status 404 returned error can't find the container with id 92298121da6c03198c569330fb1e5b586f4fb49058faf85c232667b3813aa4e4 Aug 04 21:22:29.417936 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.417885 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:29.418032 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.417967 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:29.418032 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.417995 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:29.510138 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.510097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" event={"ID":"37f07096-4393-463a-b854-1ea0c19e2df0","Type":"ContainerStarted","Data":"54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75"} Aug 04 21:22:29.510326 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.510163 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" event={"ID":"37f07096-4393-463a-b854-1ea0c19e2df0","Type":"ContainerStarted","Data":"92298121da6c03198c569330fb1e5b586f4fb49058faf85c232667b3813aa4e4"} Aug 04 21:22:29.536644 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:29.536525 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" podStartSLOduration=1.536503889 podStartE2EDuration="1.536503889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:22:29.534881489 +0000 UTC m=+2483.681752045" watchObservedRunningTime="2026-08-04 21:22:29.536503889 +0000 UTC m=+2483.683374445" Aug 04 21:22:30.270457 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:30.270411 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:30.275214 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:30.275186 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:30.514837 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:30.514807 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:30.515947 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:30.515900 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:31.440966 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:31.440927 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:31.540721 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:31.540672 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:32.559293 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:32.559255 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:32.661371 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:32.661335 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:33.086198 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:33.086160 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:33.171304 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:33.171266 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:45.516524 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:45.516485 2576 patch_prober.go:28] interesting pod/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 04 21:22:45.516994 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:45.516552 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 04 21:22:45.593441 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:45.593408 2576 generic.go:356] "Generic (PLEG): container finished" podID="37f07096-4393-463a-b854-1ea0c19e2df0" containerID="54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75" exitCode=0 Aug 04 21:22:45.593581 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:45.593467 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" event={"ID":"37f07096-4393-463a-b854-1ea0c19e2df0","Type":"ContainerDied","Data":"54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75"} Aug 04 21:22:45.593875 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:45.593858 2576 scope.go:117] "RemoveContainer" containerID="54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75" Aug 04 21:22:45.594937 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:45.594890 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:45.595012 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:45.594997 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:45.595049 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:45.595037 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:22:46.601336 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:46.601296 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" event={"ID":"37f07096-4393-463a-b854-1ea0c19e2df0","Type":"ContainerStarted","Data":"dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec"} Aug 04 21:22:47.270572 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:47.270532 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:47.270768 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:47.270744 2576 patch_prober.go:28] interesting pod/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 04 21:22:47.270820 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:47.270792 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 04 21:22:48.270297 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:48.270254 2576 patch_prober.go:28] interesting pod/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 04 21:22:48.270713 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:48.270316 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 04 21:22:49.270234 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:49.270195 2576 patch_prober.go:28] interesting pod/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 04 21:22:49.270435 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:49.270253 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 04 21:22:49.459827 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:49.459789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:22:50.270405 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:50.270362 2576 patch_prober.go:28] interesting pod/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 04 21:22:50.270599 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:50.270423 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 04 21:22:51.270431 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:51.270394 2576 patch_prober.go:28] interesting pod/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 04 21:22:51.270613 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:51.270470 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 04 21:22:52.273597 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:52.273561 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:52.274107 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:52.273943 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:52.274711 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:52.274693 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:22:56.441185 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:22:56.441142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:23:06.613881 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:23:06.613846 2576 scope.go:117] "RemoveContainer" containerID="925a66b1d44ba95ce80c7babdd173a48eb31fc9f36ca1cdbc3d404deb27a0445" Aug 04 21:23:13.357713 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:23:13.357674 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:23:33.480807 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:23:33.480772 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:16.170853 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:16.170816 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:17.105343 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.105307 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:17.206613 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.206567 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh"] Aug 04 21:24:17.209989 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.209967 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" Aug 04 21:24:17.212524 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.212498 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 21:24:17.212658 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.212639 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86svs\"" Aug 04 21:24:17.219383 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.219358 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh"] Aug 04 21:24:17.316029 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.315996 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e268df14-560e-49a1-83ae-6c035bab0a89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh\" (UID: \"e268df14-560e-49a1-83ae-6c035bab0a89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" Aug 04 21:24:17.416817 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.416738 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e268df14-560e-49a1-83ae-6c035bab0a89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh\" (UID: \"e268df14-560e-49a1-83ae-6c035bab0a89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" Aug 04 21:24:17.417362 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.417341 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e268df14-560e-49a1-83ae-6c035bab0a89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh\" (UID: \"e268df14-560e-49a1-83ae-6c035bab0a89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" Aug 04 21:24:17.522211 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.522175 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" Aug 04 21:24:17.649654 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:17.649580 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh"] Aug 04 21:24:17.652609 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:24:17.652577 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode268df14_560e_49a1_83ae_6c035bab0a89.slice/crio-90d813730c493886191d603af6594f65584e2ada858cdab40c587abcee7ee2db WatchSource:0}: Error finding container 90d813730c493886191d603af6594f65584e2ada858cdab40c587abcee7ee2db: Status 404 returned error can't find the container with id 90d813730c493886191d603af6594f65584e2ada858cdab40c587abcee7ee2db Aug 04 21:24:18.017893 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:18.017858 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" event={"ID":"e268df14-560e-49a1-83ae-6c035bab0a89","Type":"ContainerStarted","Data":"b32d651be6c473a47c340cf36615f18b2b4123e4e81fb3ced70f0ea6d3937fe3"} Aug 04 21:24:18.018065 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:18.017941 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" event={"ID":"e268df14-560e-49a1-83ae-6c035bab0a89","Type":"ContainerStarted","Data":"90d813730c493886191d603af6594f65584e2ada858cdab40c587abcee7ee2db"} Aug 04 21:24:18.034638 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:18.034585 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" podStartSLOduration=1.034569648 podStartE2EDuration="1.034569648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:24:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:24:18.032280476 +0000 UTC m=+2592.179151030" watchObservedRunningTime="2026-08-04 21:24:18.034569648 +0000 UTC m=+2592.181440233" Aug 04 21:24:19.024013 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:19.023980 2576 generic.go:356] "Generic (PLEG): container finished" podID="e268df14-560e-49a1-83ae-6c035bab0a89" containerID="b32d651be6c473a47c340cf36615f18b2b4123e4e81fb3ced70f0ea6d3937fe3" exitCode=0 Aug 04 21:24:19.024391 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:19.024064 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" event={"ID":"e268df14-560e-49a1-83ae-6c035bab0a89","Type":"ContainerDied","Data":"b32d651be6c473a47c340cf36615f18b2b4123e4e81fb3ced70f0ea6d3937fe3"} Aug 04 21:24:20.160929 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:20.160875 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" Aug 04 21:24:20.242489 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:20.242453 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e268df14-560e-49a1-83ae-6c035bab0a89-maas-api-service-ca\") pod \"e268df14-560e-49a1-83ae-6c035bab0a89\" (UID: \"e268df14-560e-49a1-83ae-6c035bab0a89\") " Aug 04 21:24:20.242826 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:20.242800 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e268df14-560e-49a1-83ae-6c035bab0a89-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e268df14-560e-49a1-83ae-6c035bab0a89" (UID: "e268df14-560e-49a1-83ae-6c035bab0a89"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:24:20.344120 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:20.344093 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e268df14-560e-49a1-83ae-6c035bab0a89-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:21.034658 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:21.034623 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" Aug 04 21:24:21.034826 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:21.034624 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh" event={"ID":"e268df14-560e-49a1-83ae-6c035bab0a89","Type":"ContainerDied","Data":"90d813730c493886191d603af6594f65584e2ada858cdab40c587abcee7ee2db"} Aug 04 21:24:21.034826 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:21.034757 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90d813730c493886191d603af6594f65584e2ada858cdab40c587abcee7ee2db" Aug 04 21:24:21.612258 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:21.612227 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:21.684467 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:21.684437 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:21.739600 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:21.739570 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:21.831363 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:21.831331 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh"] Aug 04 21:24:21.832218 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:21.832191 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:21.832607 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:21.832583 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19ad13-2t4bh"] Aug 04 21:24:22.193483 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:22.193445 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:22.439562 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:22.439531 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e268df14-560e-49a1-83ae-6c035bab0a89" path="/var/lib/kubelet/pods/e268df14-560e-49a1-83ae-6c035bab0a89/volumes" Aug 04 21:24:27.282202 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:27.282105 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:32.274731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:32.274692 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:33.264549 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:33.264513 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:33.326627 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:33.326594 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:33.529160 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:33.529082 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz"] Aug 04 21:24:33.529362 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:33.529335 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" containerID="cri-o://083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf" gracePeriod=30 Aug 04 21:24:33.546412 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:33.546387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:33.792326 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:33.792230 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:38.767476 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.767446 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:24:38.911262 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911226 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-data\") pod \"94689d38-f839-409f-89e2-534f0eac89cd\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " Aug 04 21:24:38.911474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911277 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94689d38-f839-409f-89e2-534f0eac89cd-istio-podinfo\") pod \"94689d38-f839-409f-89e2-534f0eac89cd\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " Aug 04 21:24:38.911474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911300 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-credential-socket\") pod \"94689d38-f839-409f-89e2-534f0eac89cd\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " Aug 04 21:24:38.911474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911323 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9gns\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-kube-api-access-q9gns\") pod \"94689d38-f839-409f-89e2-534f0eac89cd\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " Aug 04 21:24:38.911474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911339 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-certs\") pod \"94689d38-f839-409f-89e2-534f0eac89cd\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " Aug 04 21:24:38.911474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911381 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-istio-token\") pod \"94689d38-f839-409f-89e2-534f0eac89cd\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " Aug 04 21:24:38.911474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911399 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-envoy\") pod \"94689d38-f839-409f-89e2-534f0eac89cd\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " Aug 04 21:24:38.911474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911452 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-socket\") pod \"94689d38-f839-409f-89e2-534f0eac89cd\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " Aug 04 21:24:38.911955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911516 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94689d38-f839-409f-89e2-534f0eac89cd-istiod-ca-cert\") pod \"94689d38-f839-409f-89e2-534f0eac89cd\" (UID: \"94689d38-f839-409f-89e2-534f0eac89cd\") " Aug 04 21:24:38.911955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911598 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "94689d38-f839-409f-89e2-534f0eac89cd" (UID: "94689d38-f839-409f-89e2-534f0eac89cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:38.911955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911738 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "94689d38-f839-409f-89e2-534f0eac89cd" (UID: "94689d38-f839-409f-89e2-534f0eac89cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:38.911955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911820 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "94689d38-f839-409f-89e2-534f0eac89cd" (UID: "94689d38-f839-409f-89e2-534f0eac89cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:38.911955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911835 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:38.911955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911855 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:38.911955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.911893 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "94689d38-f839-409f-89e2-534f0eac89cd" (UID: "94689d38-f839-409f-89e2-534f0eac89cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:38.912487 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.912042 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94689d38-f839-409f-89e2-534f0eac89cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "94689d38-f839-409f-89e2-534f0eac89cd" (UID: "94689d38-f839-409f-89e2-534f0eac89cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:24:38.913732 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.913698 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-kube-api-access-q9gns" (OuterVolumeSpecName: "kube-api-access-q9gns") pod "94689d38-f839-409f-89e2-534f0eac89cd" (UID: "94689d38-f839-409f-89e2-534f0eac89cd"). InnerVolumeSpecName "kube-api-access-q9gns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:24:38.913842 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.913757 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "94689d38-f839-409f-89e2-534f0eac89cd" (UID: "94689d38-f839-409f-89e2-534f0eac89cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:38.914098 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.914071 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94689d38-f839-409f-89e2-534f0eac89cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "94689d38-f839-409f-89e2-534f0eac89cd" (UID: "94689d38-f839-409f-89e2-534f0eac89cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:24:38.914197 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:38.914072 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "94689d38-f839-409f-89e2-534f0eac89cd" (UID: "94689d38-f839-409f-89e2-534f0eac89cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:24:39.013254 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.013218 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94689d38-f839-409f-89e2-534f0eac89cd-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:39.013254 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.013245 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:39.013450 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.013269 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9gns\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-kube-api-access-q9gns\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:39.013450 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.013279 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94689d38-f839-409f-89e2-534f0eac89cd-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:39.013450 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.013287 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:39.013450 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.013297 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94689d38-f839-409f-89e2-534f0eac89cd-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:39.013450 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.013304 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94689d38-f839-409f-89e2-534f0eac89cd-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:39.126069 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.126036 2576 generic.go:356] "Generic (PLEG): container finished" podID="94689d38-f839-409f-89e2-534f0eac89cd" containerID="083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf" exitCode=0 Aug 04 21:24:39.126237 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.126104 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" Aug 04 21:24:39.126237 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.126107 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" event={"ID":"94689d38-f839-409f-89e2-534f0eac89cd","Type":"ContainerDied","Data":"083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf"} Aug 04 21:24:39.126237 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.126225 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz" event={"ID":"94689d38-f839-409f-89e2-534f0eac89cd","Type":"ContainerDied","Data":"2432932d0871d52cd75f576e1f242a25cc1b5e93266fb650b3be90227cb962cc"} Aug 04 21:24:39.126345 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.126247 2576 scope.go:117] "RemoveContainer" containerID="083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf" Aug 04 21:24:39.135367 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.135349 2576 scope.go:117] "RemoveContainer" containerID="c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5" Aug 04 21:24:39.143622 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.143600 2576 scope.go:117] "RemoveContainer" containerID="083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf" Aug 04 21:24:39.143890 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:24:39.143868 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf\": container with ID starting with 083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf not found: ID does not exist" containerID="083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf" Aug 04 21:24:39.143984 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.143895 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf"} err="failed to get container status \"083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf\": rpc error: code = NotFound desc = could not find container \"083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf\": container with ID starting with 083a1e6de0273d80ad65fedce6ff67858b38ba25bcca04a5618cab3641396daf not found: ID does not exist" Aug 04 21:24:39.143984 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.143923 2576 scope.go:117] "RemoveContainer" containerID="c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5" Aug 04 21:24:39.144168 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:24:39.144149 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5\": container with ID starting with c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5 not found: ID does not exist" containerID="c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5" Aug 04 21:24:39.144215 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.144174 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5"} err="failed to get container status \"c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5\": rpc error: code = NotFound desc = could not find container \"c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5\": container with ID starting with c61d7374f74c7b036f15e44fb08e4ba05be2e44c628d2bfea5701b277334b3f5 not found: ID does not exist" Aug 04 21:24:39.150784 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.150748 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz"] Aug 04 21:24:39.154523 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:39.154497 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-19ad13-openshift-default-584d8788d8-w77dz"] Aug 04 21:24:40.439918 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:40.439874 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94689d38-f839-409f-89e2-534f0eac89cd" path="/var/lib/kubelet/pods/94689d38-f839-409f-89e2-534f0eac89cd/volumes" Aug 04 21:24:44.512429 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:44.512391 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:44.583661 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:44.583628 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:44.656738 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:44.656707 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:44.758479 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:44.758435 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:45.049230 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:45.049199 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:51.354786 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:51.354746 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:51.455212 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:51.455168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:51.598169 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:51.598137 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl"] Aug 04 21:24:51.598405 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:51.598382 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" containerID="cri-o://dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec" gracePeriod=30 Aug 04 21:24:51.626097 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:51.626020 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:51.905750 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:51.905670 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:52.227437 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.227339 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:52.276428 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.276395 2576 patch_prober.go:28] interesting pod/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 21:24:52.276594 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.276460 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 21:24:52.519616 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.519537 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd"] Aug 04 21:24:52.519977 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.519864 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e268df14-560e-49a1-83ae-6c035bab0a89" containerName="revoke-keys" Aug 04 21:24:52.519977 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.519876 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e268df14-560e-49a1-83ae-6c035bab0a89" containerName="revoke-keys" Aug 04 21:24:52.519977 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.519898 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" Aug 04 21:24:52.519977 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.519922 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" Aug 04 21:24:52.519977 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.519936 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" Aug 04 21:24:52.519977 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.519942 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" Aug 04 21:24:52.520178 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.519998 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" Aug 04 21:24:52.520178 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.520008 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e268df14-560e-49a1-83ae-6c035bab0a89" containerName="revoke-keys" Aug 04 21:24:52.522708 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.522692 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.530721 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.530699 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a8febf-openshift-default-dockercfg-z4jxz\"" Aug 04 21:24:52.536457 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.536433 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd"] Aug 04 21:24:52.537327 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.537307 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-token\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.537431 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.537412 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-certs\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.537766 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.537444 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-data\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.537873 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.537799 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-envoy\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.537873 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.537839 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-podinfo\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.537992 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.537891 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7a50b8a-2d18-4fc6-a521-dae99931b788-istiod-ca-cert\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.538029 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.537991 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r8qc\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-kube-api-access-6r8qc\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.538064 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.538024 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-credential-socket\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.538098 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.538082 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-socket\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.542720 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.542697 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:52.638504 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638467 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-token\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.638715 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638528 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-certs\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.638715 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638635 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-data\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.638715 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-envoy\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.638945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638716 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-podinfo\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.638945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638764 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7a50b8a-2d18-4fc6-a521-dae99931b788-istiod-ca-cert\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.638945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r8qc\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-kube-api-access-6r8qc\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.638945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-credential-socket\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.638945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638851 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-certs\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.638945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.638876 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-socket\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.639191 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.639080 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-data\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.639191 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.639150 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-credential-socket\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.639258 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.639242 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-socket\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.639443 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.639419 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7a50b8a-2d18-4fc6-a521-dae99931b788-istiod-ca-cert\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.641340 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.641310 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-envoy\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.641519 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.641498 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-podinfo\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.648189 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.648165 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-token\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.648537 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.648514 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r8qc\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-kube-api-access-6r8qc\") pod \"e2e-api-a-a8febf-openshift-default-5876b54488-zspzd\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.832464 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.832391 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:52.974745 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.974715 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd"] Aug 04 21:24:52.975822 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.974853 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:24:52.975822 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.975021 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:24:52.975822 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:52.975106 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:24:53.106229 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:53.106193 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:53.195313 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:53.195276 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" event={"ID":"b7a50b8a-2d18-4fc6-a521-dae99931b788","Type":"ContainerStarted","Data":"e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c"} Aug 04 21:24:53.195497 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:53.195335 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" event={"ID":"b7a50b8a-2d18-4fc6-a521-dae99931b788","Type":"ContainerStarted","Data":"2338f2a2db4060170393934ffa42fa2f04ecc69157d2c644c471914d13c2621c"} Aug 04 21:24:53.218610 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:53.218546 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" podStartSLOduration=1.218523302 podStartE2EDuration="1.218523302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:24:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:24:53.216146871 +0000 UTC m=+2627.363017463" watchObservedRunningTime="2026-08-04 21:24:53.218523302 +0000 UTC m=+2627.365393859" Aug 04 21:24:53.832727 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:53.832693 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:54.838118 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:54.838086 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:55.150218 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:55.150179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:55.205220 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:55.205185 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:55.206199 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:55.206182 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:24:55.255382 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:55.255347 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:55.352818 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:55.352781 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:56.290637 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.290591 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:56.529520 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.529482 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:56.856244 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.856220 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:24:56.878786 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.878713 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fldvk\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-kube-api-access-fldvk\") pod \"37f07096-4393-463a-b854-1ea0c19e2df0\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " Aug 04 21:24:56.878786 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.878770 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-istio-token\") pod \"37f07096-4393-463a-b854-1ea0c19e2df0\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " Aug 04 21:24:56.879031 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.878791 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-socket\") pod \"37f07096-4393-463a-b854-1ea0c19e2df0\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " Aug 04 21:24:56.879031 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.878813 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-certs\") pod \"37f07096-4393-463a-b854-1ea0c19e2df0\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " Aug 04 21:24:56.879031 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.878832 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f07096-4393-463a-b854-1ea0c19e2df0-istio-podinfo\") pod \"37f07096-4393-463a-b854-1ea0c19e2df0\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " Aug 04 21:24:56.879031 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.878861 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-credential-socket\") pod \"37f07096-4393-463a-b854-1ea0c19e2df0\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " Aug 04 21:24:56.879031 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.878886 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f07096-4393-463a-b854-1ea0c19e2df0-istiod-ca-cert\") pod \"37f07096-4393-463a-b854-1ea0c19e2df0\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " Aug 04 21:24:56.879031 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.878964 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-envoy\") pod \"37f07096-4393-463a-b854-1ea0c19e2df0\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " Aug 04 21:24:56.879031 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.879017 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-data\") pod \"37f07096-4393-463a-b854-1ea0c19e2df0\" (UID: \"37f07096-4393-463a-b854-1ea0c19e2df0\") " Aug 04 21:24:56.879378 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.879172 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37f07096-4393-463a-b854-1ea0c19e2df0" (UID: "37f07096-4393-463a-b854-1ea0c19e2df0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:56.879378 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.879181 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37f07096-4393-463a-b854-1ea0c19e2df0" (UID: "37f07096-4393-463a-b854-1ea0c19e2df0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:56.879378 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.879203 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37f07096-4393-463a-b854-1ea0c19e2df0" (UID: "37f07096-4393-463a-b854-1ea0c19e2df0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:56.879696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.879394 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-data" (OuterVolumeSpecName: "istio-data") pod "37f07096-4393-463a-b854-1ea0c19e2df0" (UID: "37f07096-4393-463a-b854-1ea0c19e2df0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:56.879696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.879398 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:56.879696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.879415 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f07096-4393-463a-b854-1ea0c19e2df0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37f07096-4393-463a-b854-1ea0c19e2df0" (UID: "37f07096-4393-463a-b854-1ea0c19e2df0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:24:56.879696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.879439 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:56.879696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.879455 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:56.881208 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.881180 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-kube-api-access-fldvk" (OuterVolumeSpecName: "kube-api-access-fldvk") pod "37f07096-4393-463a-b854-1ea0c19e2df0" (UID: "37f07096-4393-463a-b854-1ea0c19e2df0"). InnerVolumeSpecName "kube-api-access-fldvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:24:56.881208 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.881197 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-istio-token" (OuterVolumeSpecName: "istio-token") pod "37f07096-4393-463a-b854-1ea0c19e2df0" (UID: "37f07096-4393-463a-b854-1ea0c19e2df0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:24:56.881356 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.881270 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37f07096-4393-463a-b854-1ea0c19e2df0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37f07096-4393-463a-b854-1ea0c19e2df0" (UID: "37f07096-4393-463a-b854-1ea0c19e2df0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:24:56.881356 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.881351 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37f07096-4393-463a-b854-1ea0c19e2df0" (UID: "37f07096-4393-463a-b854-1ea0c19e2df0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:24:56.973981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.973943 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:56.979960 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.979927 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fldvk\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-kube-api-access-fldvk\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:56.979960 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.979961 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f07096-4393-463a-b854-1ea0c19e2df0-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:56.980163 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.979976 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f07096-4393-463a-b854-1ea0c19e2df0-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:56.980163 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.979988 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f07096-4393-463a-b854-1ea0c19e2df0-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:56.980163 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.979998 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:56.980163 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:56.980009 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f07096-4393-463a-b854-1ea0c19e2df0-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:24:57.088501 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.088465 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:24:57.217554 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.217452 2576 generic.go:356] "Generic (PLEG): container finished" podID="37f07096-4393-463a-b854-1ea0c19e2df0" containerID="dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec" exitCode=0 Aug 04 21:24:57.217554 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.217513 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" Aug 04 21:24:57.217776 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.217552 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" event={"ID":"37f07096-4393-463a-b854-1ea0c19e2df0","Type":"ContainerDied","Data":"dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec"} Aug 04 21:24:57.217776 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.217631 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl" event={"ID":"37f07096-4393-463a-b854-1ea0c19e2df0","Type":"ContainerDied","Data":"92298121da6c03198c569330fb1e5b586f4fb49058faf85c232667b3813aa4e4"} Aug 04 21:24:57.217776 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.217667 2576 scope.go:117] "RemoveContainer" containerID="dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec" Aug 04 21:24:57.243935 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.243886 2576 scope.go:117] "RemoveContainer" containerID="54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75" Aug 04 21:24:57.252255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.252235 2576 scope.go:117] "RemoveContainer" containerID="dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec" Aug 04 21:24:57.252498 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:24:57.252479 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec\": container with ID starting with dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec not found: ID does not exist" containerID="dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec" Aug 04 21:24:57.252559 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.252507 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec"} err="failed to get container status \"dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec\": rpc error: code = NotFound desc = could not find container \"dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec\": container with ID starting with dbad468269e2088f0079dc88fa45250ae8cbba1a6f9ae00e1e9383e5668279ec not found: ID does not exist" Aug 04 21:24:57.252559 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.252522 2576 scope.go:117] "RemoveContainer" containerID="54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75" Aug 04 21:24:57.252751 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:24:57.252732 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75\": container with ID starting with 54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75 not found: ID does not exist" containerID="54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75" Aug 04 21:24:57.252793 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.252758 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75"} err="failed to get container status \"54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75\": rpc error: code = NotFound desc = could not find container \"54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75\": container with ID starting with 54765c45909289dcba00178b1aea1e722b509baa4c4af8cacddd95aafaa15c75 not found: ID does not exist" Aug 04 21:24:57.260800 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.260769 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl"] Aug 04 21:24:57.262545 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:57.262523 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4c3747-openshift-default-fb77c6456-ftzkl"] Aug 04 21:24:58.439683 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:24:58.439652 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" path="/var/lib/kubelet/pods/37f07096-4393-463a-b854-1ea0c19e2df0/volumes" Aug 04 21:25:09.282700 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:09.282593 2576 generic.go:356] "Generic (PLEG): container finished" podID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerID="e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c" exitCode=0 Aug 04 21:25:09.282700 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:09.282665 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" event={"ID":"b7a50b8a-2d18-4fc6-a521-dae99931b788","Type":"ContainerDied","Data":"e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c"} Aug 04 21:25:09.283172 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:09.283097 2576 scope.go:117] "RemoveContainer" containerID="e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c" Aug 04 21:25:09.284077 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:09.284051 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:25:09.284142 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:09.284119 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:25:09.284178 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:09.284151 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:25:10.205237 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:10.205198 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:25:10.289474 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:10.289440 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" event={"ID":"b7a50b8a-2d18-4fc6-a521-dae99931b788","Type":"ContainerStarted","Data":"af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f"} Aug 04 21:25:10.834494 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:10.833444 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:25:10.834840 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:10.834810 2576 patch_prober.go:28] interesting pod/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Aug 04 21:25:10.835071 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:10.835044 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Aug 04 21:25:11.833129 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:11.833092 2576 patch_prober.go:28] interesting pod/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Aug 04 21:25:11.833722 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:11.833691 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Aug 04 21:25:12.834340 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:12.834297 2576 patch_prober.go:28] interesting pod/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Aug 04 21:25:12.834983 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:12.834945 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Aug 04 21:25:13.833055 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:13.833013 2576 patch_prober.go:28] interesting pod/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Aug 04 21:25:13.833358 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:13.833323 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Aug 04 21:25:14.833427 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:14.833386 2576 patch_prober.go:28] interesting pod/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Aug 04 21:25:14.834040 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:14.834008 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Aug 04 21:25:15.833813 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:15.833765 2576 patch_prober.go:28] interesting pod/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Aug 04 21:25:15.834247 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:15.833833 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Aug 04 21:25:16.837454 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:16.837420 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:25:16.837837 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:16.837698 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:25:16.838464 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:16.838449 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:25:20.122803 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.122773 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:20.593441 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.593410 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:20.863408 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.863330 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x"] Aug 04 21:25:20.863781 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.863765 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" Aug 04 21:25:20.863834 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.863786 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" Aug 04 21:25:20.863834 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.863802 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" Aug 04 21:25:20.863834 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.863808 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" Aug 04 21:25:20.863987 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.863867 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="94689d38-f839-409f-89e2-534f0eac89cd" containerName="istio-proxy" Aug 04 21:25:20.863987 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.863877 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" Aug 04 21:25:20.863987 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.863883 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f07096-4393-463a-b854-1ea0c19e2df0" containerName="istio-proxy" Aug 04 21:25:20.866220 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.866202 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:20.868718 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.868699 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-c4ea15-openshift-default-dockercfg-lnjxv\"" Aug 04 21:25:20.881096 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:20.881071 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x"] Aug 04 21:25:21.008157 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.008093 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:21.019940 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.019887 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-socket\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.020128 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.020054 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2jfr\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-kube-api-access-v2jfr\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.020194 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.020149 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-certs\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.020194 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.020180 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7834d7a-655c-4e05-98bb-671b35dbfd33-istiod-ca-cert\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.020297 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.020200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-data\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.020297 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.020234 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-podinfo\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.020405 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.020315 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-credential-socket\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.020405 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.020348 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-token\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.020405 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.020381 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-envoy\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.121542 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121459 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-socket\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.121542 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121496 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2jfr\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-kube-api-access-v2jfr\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.121542 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121524 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-certs\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.121767 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7834d7a-655c-4e05-98bb-671b35dbfd33-istiod-ca-cert\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.121767 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121666 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-data\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.121767 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121707 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-podinfo\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.121767 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121753 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-credential-socket\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.122082 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121800 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-token\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.122082 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121836 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-envoy\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.122082 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121943 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-socket\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.122082 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.121980 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-certs\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.122255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.122098 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-data\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.122255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.122173 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-credential-socket\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.122361 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.122333 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7834d7a-655c-4e05-98bb-671b35dbfd33-istiod-ca-cert\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.123975 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.123949 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:21.124483 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.124461 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-envoy\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.124734 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.124717 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-podinfo\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.129980 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.129952 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2jfr\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-kube-api-access-v2jfr\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.130402 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.130381 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-token\") pod \"e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.176498 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.176462 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:21.310386 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.310356 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x"] Aug 04 21:25:21.312942 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:25:21.312897 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7834d7a_655c_4e05_98bb_671b35dbfd33.slice/crio-9364dbeb7cea112e83903246b52c6ab038f159d5b7a43cff45e19adf7ee53b8a WatchSource:0}: Error finding container 9364dbeb7cea112e83903246b52c6ab038f159d5b7a43cff45e19adf7ee53b8a: Status 404 returned error can't find the container with id 9364dbeb7cea112e83903246b52c6ab038f159d5b7a43cff45e19adf7ee53b8a Aug 04 21:25:21.315134 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.315103 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:25:21.315217 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.315164 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:25:21.315217 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.315193 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:25:21.340257 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:21.340229 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" event={"ID":"d7834d7a-655c-4e05-98bb-671b35dbfd33","Type":"ContainerStarted","Data":"9364dbeb7cea112e83903246b52c6ab038f159d5b7a43cff45e19adf7ee53b8a"} Aug 04 21:25:22.346093 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:22.346054 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" event={"ID":"d7834d7a-655c-4e05-98bb-671b35dbfd33","Type":"ContainerStarted","Data":"dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36"} Aug 04 21:25:22.369092 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:22.369020 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" podStartSLOduration=2.368998899 podStartE2EDuration="2.368998899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:25:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:25:22.365817268 +0000 UTC m=+2656.512687805" watchObservedRunningTime="2026-08-04 21:25:22.368998899 +0000 UTC m=+2656.515869456" Aug 04 21:25:23.176925 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:23.176867 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:23.181988 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:23.181960 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:23.351059 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:23.351028 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:23.352202 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:23.352184 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:24.392897 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:24.392866 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:24.494064 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:24.494031 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:25.528530 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:25.528497 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:25.733564 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:25.733525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:26.127058 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:26.127021 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:26.225230 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:26.225195 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:25:38.352730 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:38.352691 2576 patch_prober.go:28] interesting pod/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Aug 04 21:25:38.353196 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:38.352762 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Aug 04 21:25:38.423778 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:38.423745 2576 generic.go:356] "Generic (PLEG): container finished" podID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerID="dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36" exitCode=0 Aug 04 21:25:38.423983 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:38.423820 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" event={"ID":"d7834d7a-655c-4e05-98bb-671b35dbfd33","Type":"ContainerDied","Data":"dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36"} Aug 04 21:25:38.424264 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:38.424249 2576 scope.go:117] "RemoveContainer" containerID="dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36" Aug 04 21:25:38.425261 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:38.425234 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:25:38.425320 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:38.425300 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:25:38.425363 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:38.425333 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:25:39.435118 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:39.435075 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" event={"ID":"d7834d7a-655c-4e05-98bb-671b35dbfd33","Type":"ContainerStarted","Data":"600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd"} Aug 04 21:25:40.177139 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:40.177098 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:40.177422 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:40.177402 2576 patch_prober.go:28] interesting pod/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Aug 04 21:25:40.177488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:40.177455 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Aug 04 21:25:41.177788 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:41.177751 2576 patch_prober.go:28] interesting pod/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Aug 04 21:25:41.178210 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:41.177816 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Aug 04 21:25:42.177569 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:42.177527 2576 patch_prober.go:28] interesting pod/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Aug 04 21:25:42.177758 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:42.177591 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Aug 04 21:25:43.177899 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:43.177858 2576 patch_prober.go:28] interesting pod/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Aug 04 21:25:43.178337 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:43.177951 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Aug 04 21:25:44.177018 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:44.176973 2576 patch_prober.go:28] interesting pod/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Aug 04 21:25:44.177239 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:44.177054 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Aug 04 21:25:45.181210 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:45.181173 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:45.181694 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:45.181662 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:45.182386 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:45.182364 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:25:49.299575 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:25:49.299538 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:02.068399 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:02.068309 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:03.294776 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:03.294736 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:06.656645 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:06.656611 2576 scope.go:117] "RemoveContainer" containerID="60db6bfa58a9472d16fd16eb77a36d9dfb7798eac734b29a274b4783fe6bed1f" Aug 04 21:26:08.424392 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:08.424352 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:08.522976 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:08.522942 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:08.644561 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:08.644527 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd"] Aug 04 21:26:08.644763 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:08.644742 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" containerID="cri-o://af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f" gracePeriod=30 Aug 04 21:26:08.666189 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:08.666145 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:08.927055 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:08.927018 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:13.892408 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.892380 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:26:13.918181 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918096 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-socket\") pod \"b7a50b8a-2d18-4fc6-a521-dae99931b788\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " Aug 04 21:26:13.918181 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918136 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-data\") pod \"b7a50b8a-2d18-4fc6-a521-dae99931b788\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " Aug 04 21:26:13.918181 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918155 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-certs\") pod \"b7a50b8a-2d18-4fc6-a521-dae99931b788\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " Aug 04 21:26:13.918181 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918171 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-envoy\") pod \"b7a50b8a-2d18-4fc6-a521-dae99931b788\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " Aug 04 21:26:13.918508 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918212 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-token\") pod \"b7a50b8a-2d18-4fc6-a521-dae99931b788\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " Aug 04 21:26:13.918508 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918233 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-podinfo\") pod \"b7a50b8a-2d18-4fc6-a521-dae99931b788\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " Aug 04 21:26:13.918508 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918260 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r8qc\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-kube-api-access-6r8qc\") pod \"b7a50b8a-2d18-4fc6-a521-dae99931b788\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " Aug 04 21:26:13.918508 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918289 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-credential-socket\") pod \"b7a50b8a-2d18-4fc6-a521-dae99931b788\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " Aug 04 21:26:13.918508 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918399 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7a50b8a-2d18-4fc6-a521-dae99931b788" (UID: "b7a50b8a-2d18-4fc6-a521-dae99931b788"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:13.918508 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918415 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7a50b8a-2d18-4fc6-a521-dae99931b788" (UID: "b7a50b8a-2d18-4fc6-a521-dae99931b788"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:13.918508 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918480 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7a50b8a-2d18-4fc6-a521-dae99931b788" (UID: "b7a50b8a-2d18-4fc6-a521-dae99931b788"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:13.918849 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918572 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:13.918849 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918585 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:13.918849 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918593 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:13.918849 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.918594 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7a50b8a-2d18-4fc6-a521-dae99931b788" (UID: "b7a50b8a-2d18-4fc6-a521-dae99931b788"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:13.921150 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.921124 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7a50b8a-2d18-4fc6-a521-dae99931b788" (UID: "b7a50b8a-2d18-4fc6-a521-dae99931b788"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:26:13.921274 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.921213 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7a50b8a-2d18-4fc6-a521-dae99931b788" (UID: "b7a50b8a-2d18-4fc6-a521-dae99931b788"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:13.921274 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.921247 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7a50b8a-2d18-4fc6-a521-dae99931b788" (UID: "b7a50b8a-2d18-4fc6-a521-dae99931b788"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:26:13.921363 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:13.921297 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-kube-api-access-6r8qc" (OuterVolumeSpecName: "kube-api-access-6r8qc") pod "b7a50b8a-2d18-4fc6-a521-dae99931b788" (UID: "b7a50b8a-2d18-4fc6-a521-dae99931b788"). InnerVolumeSpecName "kube-api-access-6r8qc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:26:14.019378 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.019338 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7a50b8a-2d18-4fc6-a521-dae99931b788-istiod-ca-cert\") pod \"b7a50b8a-2d18-4fc6-a521-dae99931b788\" (UID: \"b7a50b8a-2d18-4fc6-a521-dae99931b788\") " Aug 04 21:26:14.019615 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.019600 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:14.019696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.019620 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:14.019696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.019633 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7a50b8a-2d18-4fc6-a521-dae99931b788-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:14.019696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.019644 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r8qc\" (UniqueName: \"kubernetes.io/projected/b7a50b8a-2d18-4fc6-a521-dae99931b788-kube-api-access-6r8qc\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:14.019696 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.019653 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7a50b8a-2d18-4fc6-a521-dae99931b788-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:14.019869 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.019758 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7a50b8a-2d18-4fc6-a521-dae99931b788-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7a50b8a-2d18-4fc6-a521-dae99931b788" (UID: "b7a50b8a-2d18-4fc6-a521-dae99931b788"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:26:14.121138 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.121094 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7a50b8a-2d18-4fc6-a521-dae99931b788-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:14.608066 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.608032 2576 generic.go:356] "Generic (PLEG): container finished" podID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerID="af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f" exitCode=0 Aug 04 21:26:14.608257 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.608088 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" Aug 04 21:26:14.608257 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.608102 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" event={"ID":"b7a50b8a-2d18-4fc6-a521-dae99931b788","Type":"ContainerDied","Data":"af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f"} Aug 04 21:26:14.608257 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.608145 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd" event={"ID":"b7a50b8a-2d18-4fc6-a521-dae99931b788","Type":"ContainerDied","Data":"2338f2a2db4060170393934ffa42fa2f04ecc69157d2c644c471914d13c2621c"} Aug 04 21:26:14.608257 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.608165 2576 scope.go:117] "RemoveContainer" containerID="af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f" Aug 04 21:26:14.617411 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.617295 2576 scope.go:117] "RemoveContainer" containerID="e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c" Aug 04 21:26:14.626190 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.625949 2576 scope.go:117] "RemoveContainer" containerID="af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f" Aug 04 21:26:14.626326 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:26:14.626258 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f\": container with ID starting with af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f not found: ID does not exist" containerID="af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f" Aug 04 21:26:14.626326 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.626292 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f"} err="failed to get container status \"af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f\": rpc error: code = NotFound desc = could not find container \"af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f\": container with ID starting with af18a473d39dda6acf70032dd5a85b39e9759f32a9748793f0ddb967267bd29f not found: ID does not exist" Aug 04 21:26:14.626326 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.626311 2576 scope.go:117] "RemoveContainer" containerID="e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c" Aug 04 21:26:14.626638 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:26:14.626616 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c\": container with ID starting with e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c not found: ID does not exist" containerID="e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c" Aug 04 21:26:14.626742 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.626642 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c"} err="failed to get container status \"e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c\": rpc error: code = NotFound desc = could not find container \"e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c\": container with ID starting with e8dea5569b0ab32b4e57097113087b22f3b874120d4b043db9139b9bad13eb2c not found: ID does not exist" Aug 04 21:26:14.627422 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.627404 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd"] Aug 04 21:26:14.631405 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:14.631382 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a8febf-openshift-default-5876b54488-zspzd"] Aug 04 21:26:16.440538 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:16.440503 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" path="/var/lib/kubelet/pods/b7a50b8a-2d18-4fc6-a521-dae99931b788/volumes" Aug 04 21:26:19.686022 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:19.685978 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:24.616680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:24.616636 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:24.685838 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:24.685795 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:24.744526 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:24.744488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:24.802678 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:24.802641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:25.184561 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:25.184526 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:31.362547 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:31.362512 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:31.442296 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:31.442257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:31.654695 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:31.654596 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x"] Aug 04 21:26:31.654960 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:31.654932 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" containerID="cri-o://600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd" gracePeriod=30 Aug 04 21:26:31.689473 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:31.689441 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:31.918871 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:31.918780 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:32.253484 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.253357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:32.529191 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.529106 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk"] Aug 04 21:26:32.529628 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.529605 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" Aug 04 21:26:32.529708 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.529633 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" Aug 04 21:26:32.529708 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.529646 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" Aug 04 21:26:32.529708 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.529654 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" Aug 04 21:26:32.529870 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.529767 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" Aug 04 21:26:32.529870 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.529784 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a50b8a-2d18-4fc6-a521-dae99931b788" containerName="istio-proxy" Aug 04 21:26:32.534362 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.534342 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.536968 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.536947 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a189ec-openshift-default-dockercfg-zdv9w\"" Aug 04 21:26:32.550897 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.550864 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk"] Aug 04 21:26:32.558691 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.558656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:32.584583 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.584551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-credential-socket\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.584583 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.584584 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-certs\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.584805 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.584607 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-data\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.584805 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.584629 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14af029-8b30-4c67-9f2f-9d29cc559943-istiod-ca-cert\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.584805 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.584775 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-envoy\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.585019 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.584826 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-token\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.585019 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.584871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-socket\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.585019 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.584898 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-podinfo\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.585019 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.584963 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwwhz\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-kube-api-access-vwwhz\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.685752 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.685710 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-envoy\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.685934 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.685761 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-token\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.685934 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.685893 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-socket\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686066 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.685957 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-podinfo\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686066 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.685980 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwwhz\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-kube-api-access-vwwhz\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686066 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.686021 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-credential-socket\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686234 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.686168 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-certs\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686234 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.686220 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-data\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686341 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.686259 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14af029-8b30-4c67-9f2f-9d29cc559943-istiod-ca-cert\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686341 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.686300 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-socket\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686451 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.686355 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-credential-socket\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686526 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.686504 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-certs\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686563 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.686525 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-data\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.686789 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.686771 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14af029-8b30-4c67-9f2f-9d29cc559943-istiod-ca-cert\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.688483 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.688461 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-envoy\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.688608 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.688592 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-podinfo\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.697562 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.697539 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwwhz\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-kube-api-access-vwwhz\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.697662 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.697545 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-token\") pod \"e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.766131 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.766094 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:32.851056 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.851031 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:32.989872 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.989847 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk"] Aug 04 21:26:32.992219 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:26:32.992190 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf14af029_8b30_4c67_9f2f_9d29cc559943.slice/crio-627fafa573d951b9effa7ea1f6dfeeb370a367446306d0c0ac26ba98e31e2c71 WatchSource:0}: Error finding container 627fafa573d951b9effa7ea1f6dfeeb370a367446306d0c0ac26ba98e31e2c71: Status 404 returned error can't find the container with id 627fafa573d951b9effa7ea1f6dfeeb370a367446306d0c0ac26ba98e31e2c71 Aug 04 21:26:32.994223 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.994193 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:26:32.994288 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.994257 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:26:32.994332 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:32.994286 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:26:33.699455 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:33.699421 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" event={"ID":"f14af029-8b30-4c67-9f2f-9d29cc559943","Type":"ContainerStarted","Data":"d59ae457e40a01294d535780052c787d7a58e3ee06a5739116d7172aad88aa0c"} Aug 04 21:26:33.699806 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:33.699475 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" event={"ID":"f14af029-8b30-4c67-9f2f-9d29cc559943","Type":"ContainerStarted","Data":"627fafa573d951b9effa7ea1f6dfeeb370a367446306d0c0ac26ba98e31e2c71"} Aug 04 21:26:33.720515 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:33.720467 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" podStartSLOduration=1.7204494590000001 podStartE2EDuration="1.720449459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:26:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:26:33.719689701 +0000 UTC m=+2727.866560256" watchObservedRunningTime="2026-08-04 21:26:33.720449459 +0000 UTC m=+2727.867320015" Aug 04 21:26:33.852101 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:33.852065 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:34.855893 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:34.855859 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:35.607000 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:35.606963 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:35.705233 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:35.705203 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:35.708570 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:35.708541 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:35.709596 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:35.709578 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:36.743087 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:36.743048 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:36.899940 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:36.899898 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:26:37.024729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.024643 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2jfr\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-kube-api-access-v2jfr\") pod \"d7834d7a-655c-4e05-98bb-671b35dbfd33\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " Aug 04 21:26:37.024729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.024679 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-podinfo\") pod \"d7834d7a-655c-4e05-98bb-671b35dbfd33\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " Aug 04 21:26:37.024729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.024723 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-data\") pod \"d7834d7a-655c-4e05-98bb-671b35dbfd33\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " Aug 04 21:26:37.025061 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.024776 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-envoy\") pod \"d7834d7a-655c-4e05-98bb-671b35dbfd33\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " Aug 04 21:26:37.025061 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.024795 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7834d7a-655c-4e05-98bb-671b35dbfd33-istiod-ca-cert\") pod \"d7834d7a-655c-4e05-98bb-671b35dbfd33\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " Aug 04 21:26:37.025061 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.024819 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-credential-socket\") pod \"d7834d7a-655c-4e05-98bb-671b35dbfd33\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " Aug 04 21:26:37.025061 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.024845 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-socket\") pod \"d7834d7a-655c-4e05-98bb-671b35dbfd33\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " Aug 04 21:26:37.025061 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.024860 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-certs\") pod \"d7834d7a-655c-4e05-98bb-671b35dbfd33\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " Aug 04 21:26:37.025061 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.024896 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-token\") pod \"d7834d7a-655c-4e05-98bb-671b35dbfd33\" (UID: \"d7834d7a-655c-4e05-98bb-671b35dbfd33\") " Aug 04 21:26:37.025499 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.025371 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7834d7a-655c-4e05-98bb-671b35dbfd33" (UID: "d7834d7a-655c-4e05-98bb-671b35dbfd33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:37.025499 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.025432 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7834d7a-655c-4e05-98bb-671b35dbfd33-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7834d7a-655c-4e05-98bb-671b35dbfd33" (UID: "d7834d7a-655c-4e05-98bb-671b35dbfd33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:26:37.025499 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.025607 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7834d7a-655c-4e05-98bb-671b35dbfd33" (UID: "d7834d7a-655c-4e05-98bb-671b35dbfd33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:37.025499 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.025633 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7834d7a-655c-4e05-98bb-671b35dbfd33" (UID: "d7834d7a-655c-4e05-98bb-671b35dbfd33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:37.026009 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.025707 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7834d7a-655c-4e05-98bb-671b35dbfd33" (UID: "d7834d7a-655c-4e05-98bb-671b35dbfd33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:37.027346 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.027321 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7834d7a-655c-4e05-98bb-671b35dbfd33" (UID: "d7834d7a-655c-4e05-98bb-671b35dbfd33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:26:37.027468 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.027353 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-kube-api-access-v2jfr" (OuterVolumeSpecName: "kube-api-access-v2jfr") pod "d7834d7a-655c-4e05-98bb-671b35dbfd33" (UID: "d7834d7a-655c-4e05-98bb-671b35dbfd33"). InnerVolumeSpecName "kube-api-access-v2jfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:26:37.027733 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.027710 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7834d7a-655c-4e05-98bb-671b35dbfd33" (UID: "d7834d7a-655c-4e05-98bb-671b35dbfd33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:26:37.027825 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.027802 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7834d7a-655c-4e05-98bb-671b35dbfd33" (UID: "d7834d7a-655c-4e05-98bb-671b35dbfd33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:26:37.125731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.125698 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:37.125731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.125726 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7834d7a-655c-4e05-98bb-671b35dbfd33-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:37.125731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.125736 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:37.126001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.125746 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:37.126001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.125755 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:37.126001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.125763 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:37.126001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.125771 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2jfr\" (UniqueName: \"kubernetes.io/projected/d7834d7a-655c-4e05-98bb-671b35dbfd33-kube-api-access-v2jfr\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:37.126001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.125779 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:37.126001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.125787 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7834d7a-655c-4e05-98bb-671b35dbfd33-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:26:37.256610 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.256554 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:37.352292 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.352254 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:37.719929 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.719813 2576 generic.go:356] "Generic (PLEG): container finished" podID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerID="600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd" exitCode=0 Aug 04 21:26:37.719929 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.719870 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" Aug 04 21:26:37.720156 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.719898 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" event={"ID":"d7834d7a-655c-4e05-98bb-671b35dbfd33","Type":"ContainerDied","Data":"600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd"} Aug 04 21:26:37.720156 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.719984 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x" event={"ID":"d7834d7a-655c-4e05-98bb-671b35dbfd33","Type":"ContainerDied","Data":"9364dbeb7cea112e83903246b52c6ab038f159d5b7a43cff45e19adf7ee53b8a"} Aug 04 21:26:37.720156 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.720008 2576 scope.go:117] "RemoveContainer" containerID="600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd" Aug 04 21:26:37.729307 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.729288 2576 scope.go:117] "RemoveContainer" containerID="dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36" Aug 04 21:26:37.737046 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.737029 2576 scope.go:117] "RemoveContainer" containerID="600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd" Aug 04 21:26:37.737278 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:26:37.737263 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd\": container with ID starting with 600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd not found: ID does not exist" containerID="600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd" Aug 04 21:26:37.737316 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.737286 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd"} err="failed to get container status \"600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd\": rpc error: code = NotFound desc = could not find container \"600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd\": container with ID starting with 600d17a29add76dbe53883a7510bf9aab72382b43238e05db67f882adf3151fd not found: ID does not exist" Aug 04 21:26:37.737316 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.737306 2576 scope.go:117] "RemoveContainer" containerID="dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36" Aug 04 21:26:37.737517 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:26:37.737501 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36\": container with ID starting with dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36 not found: ID does not exist" containerID="dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36" Aug 04 21:26:37.737552 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.737521 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36"} err="failed to get container status \"dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36\": rpc error: code = NotFound desc = could not find container \"dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36\": container with ID starting with dd3c9ddf8ff6e181e8d4590ab077fc9efd5ace8387a915ebbcf2a468701bff36 not found: ID does not exist" Aug 04 21:26:37.742337 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.742314 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x"] Aug 04 21:26:37.745853 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.745833 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-c4ea15-openshift-default-644d58c868-qtx4x"] Aug 04 21:26:37.758516 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:37.758492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:26:38.441132 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:38.441100 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" path="/var/lib/kubelet/pods/d7834d7a-655c-4e05-98bb-671b35dbfd33/volumes" Aug 04 21:26:49.784874 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:49.784839 2576 generic.go:356] "Generic (PLEG): container finished" podID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerID="d59ae457e40a01294d535780052c787d7a58e3ee06a5739116d7172aad88aa0c" exitCode=0 Aug 04 21:26:49.785317 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:49.784931 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" event={"ID":"f14af029-8b30-4c67-9f2f-9d29cc559943","Type":"ContainerDied","Data":"d59ae457e40a01294d535780052c787d7a58e3ee06a5739116d7172aad88aa0c"} Aug 04 21:26:49.785360 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:49.785329 2576 scope.go:117] "RemoveContainer" containerID="d59ae457e40a01294d535780052c787d7a58e3ee06a5739116d7172aad88aa0c" Aug 04 21:26:49.786373 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:49.786345 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:26:49.786447 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:49.786430 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:26:49.786493 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:49.786479 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:26:50.708840 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:50.708805 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:50.791153 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:50.791116 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" event={"ID":"f14af029-8b30-4c67-9f2f-9d29cc559943","Type":"ContainerStarted","Data":"52bbe62fd03716c79bf7942cbe9aa0671b4572605c37f103cc851b9189631013"} Aug 04 21:26:50.851725 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:50.851659 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:50.852971 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:50.852945 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 21:26:50.853119 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:50.852996 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 21:26:51.852404 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:51.852362 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 21:26:51.852805 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:51.852438 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 21:26:52.852276 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:52.852230 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 21:26:52.852468 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:52.852309 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 21:26:53.852194 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:53.852152 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 21:26:53.852384 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:53.852229 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 21:26:54.852313 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:54.852267 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 21:26:54.852725 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:54.852330 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 21:26:55.851459 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:55.851422 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 21:26:55.851649 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:55.851496 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 21:26:56.855629 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:56.855595 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:56.856020 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:56.855879 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:26:56.856637 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:26:56.856619 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:27:00.595953 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:00.595891 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:00.935432 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:00.935340 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:01.248886 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.248802 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp"] Aug 04 21:27:01.249380 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.249358 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" Aug 04 21:27:01.249434 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.249386 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" Aug 04 21:27:01.249434 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.249406 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" Aug 04 21:27:01.249434 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.249413 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" Aug 04 21:27:01.249573 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.249483 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" Aug 04 21:27:01.252751 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.252727 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.256136 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.256117 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-cd644a-openshift-default-dockercfg-c22ld\"" Aug 04 21:27:01.265533 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.265498 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp"] Aug 04 21:27:01.294428 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.294387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:01.341845 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.341782 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-data\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.342075 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.341853 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-token\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.342075 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.341930 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-credential-socket\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.342075 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.341958 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-certs\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.342075 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.341998 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-socket\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.342075 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.342072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr72j\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-kube-api-access-sr72j\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.342311 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.342123 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-podinfo\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.342311 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.342148 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-envoy\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.342311 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.342165 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5098c4de-c711-488a-a0ee-9a6c7da291b5-istiod-ca-cert\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443151 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443110 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-socket\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443170 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr72j\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-kube-api-access-sr72j\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443207 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-podinfo\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-envoy\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443243 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5098c4de-c711-488a-a0ee-9a6c7da291b5-istiod-ca-cert\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443270 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-data\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443298 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-token\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443342 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-credential-socket\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443772 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443364 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-certs\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443772 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443722 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-certs\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443767 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-socket\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443791 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-credential-socket\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.443880 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443825 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-data\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.444228 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.443951 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5098c4de-c711-488a-a0ee-9a6c7da291b5-istiod-ca-cert\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.445643 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.445621 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-envoy\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.445858 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.445839 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-podinfo\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.452219 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.452193 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-token\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.452219 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.452205 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr72j\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-kube-api-access-sr72j\") pod \"e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.504216 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.504114 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:01.562736 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.562703 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:01.701851 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.701821 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp"] Aug 04 21:27:01.704410 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:27:01.704381 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5098c4de_c711_488a_a0ee_9a6c7da291b5.slice/crio-640e85e7985be5f7020fb785e61674a32970da6d0100b7898a05519e0fc45f65 WatchSource:0}: Error finding container 640e85e7985be5f7020fb785e61674a32970da6d0100b7898a05519e0fc45f65: Status 404 returned error can't find the container with id 640e85e7985be5f7020fb785e61674a32970da6d0100b7898a05519e0fc45f65 Aug 04 21:27:01.706796 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.706764 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:27:01.706887 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.706828 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:27:01.706887 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.706855 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:27:01.844348 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.844311 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" event={"ID":"5098c4de-c711-488a-a0ee-9a6c7da291b5","Type":"ContainerStarted","Data":"34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379"} Aug 04 21:27:01.844561 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.844369 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" event={"ID":"5098c4de-c711-488a-a0ee-9a6c7da291b5","Type":"ContainerStarted","Data":"640e85e7985be5f7020fb785e61674a32970da6d0100b7898a05519e0fc45f65"} Aug 04 21:27:01.864308 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:01.864189 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" podStartSLOduration=0.864167432 podStartE2EDuration="864.167432ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:27:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:27:01.863343082 +0000 UTC m=+2756.010213680" watchObservedRunningTime="2026-08-04 21:27:01.864167432 +0000 UTC m=+2756.011037988" Aug 04 21:27:02.563668 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:02.563627 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:02.569007 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:02.568979 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:02.849289 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:02.849258 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:02.850224 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:02.850203 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:03.883522 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:03.883485 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:03.974281 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:03.974243 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:04.982662 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:04.982620 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:05.220057 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:05.220020 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:05.589265 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:05.589225 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:05.695192 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:05.695151 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:17.850945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:17.850885 2576 patch_prober.go:28] interesting pod/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Aug 04 21:27:17.851429 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:17.850972 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Aug 04 21:27:17.918138 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:17.918105 2576 generic.go:356] "Generic (PLEG): container finished" podID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerID="34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379" exitCode=0 Aug 04 21:27:17.918304 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:17.918178 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" event={"ID":"5098c4de-c711-488a-a0ee-9a6c7da291b5","Type":"ContainerDied","Data":"34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379"} Aug 04 21:27:17.918547 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:17.918534 2576 scope.go:117] "RemoveContainer" containerID="34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379" Aug 04 21:27:17.919521 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:17.919495 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:27:17.919574 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:17.919563 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:27:17.919608 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:17.919592 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:27:18.924881 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:18.924843 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" event={"ID":"5098c4de-c711-488a-a0ee-9a6c7da291b5","Type":"ContainerStarted","Data":"9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130"} Aug 04 21:27:19.563553 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:19.563515 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:19.563766 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:19.563717 2576 patch_prober.go:28] interesting pod/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Aug 04 21:27:19.563822 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:19.563769 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Aug 04 21:27:20.564151 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:20.564113 2576 patch_prober.go:28] interesting pod/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Aug 04 21:27:20.564547 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:20.564187 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Aug 04 21:27:21.563567 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:21.563526 2576 patch_prober.go:28] interesting pod/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Aug 04 21:27:21.563751 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:21.563601 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Aug 04 21:27:22.563327 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:22.563281 2576 patch_prober.go:28] interesting pod/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Aug 04 21:27:22.563744 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:22.563343 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Aug 04 21:27:23.563550 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:23.563509 2576 patch_prober.go:28] interesting pod/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Aug 04 21:27:23.564003 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:23.563573 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Aug 04 21:27:24.567396 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:24.567366 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:24.567874 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:24.567607 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:24.568401 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:24.568386 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:27:28.781358 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:28.781265 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:35.363498 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:35.363446 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:27:39.276144 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:27:39.276102 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:28:02.696542 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:28:02.696505 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:28:58.366253 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:28:58.366158 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:29:13.246850 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:29:13.246813 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:29:44.838983 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:29:44.838946 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:30:00.138006 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.137962 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc"] Aug 04 21:30:00.138482 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.138377 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7834d7a-655c-4e05-98bb-671b35dbfd33" containerName="istio-proxy" Aug 04 21:30:00.141639 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.141614 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv"] Aug 04 21:30:00.141782 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.141754 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc" Aug 04 21:30:00.144716 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.144693 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p"] Aug 04 21:30:00.144867 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.144809 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv" Aug 04 21:30:00.145775 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.145753 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86svs\"" Aug 04 21:30:00.147745 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.147724 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p" Aug 04 21:30:00.152719 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.152697 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc" Aug 04 21:30:00.160127 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.160108 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv" Aug 04 21:30:00.173313 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.173234 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv"] Aug 04 21:30:00.181516 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.181412 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc"] Aug 04 21:30:00.181516 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.181481 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p"] Aug 04 21:30:00.203442 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.203412 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p" Aug 04 21:30:00.344252 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.344219 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc"] Aug 04 21:30:00.346216 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:30:00.346184 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02c9e4e_e482_4810_92e6_8f481d04a077.slice/crio-deec87834888193427673827c3c3f862825330c1b39102771f2227b225389301 WatchSource:0}: Error finding container deec87834888193427673827c3c3f862825330c1b39102771f2227b225389301: Status 404 returned error can't find the container with id deec87834888193427673827c3c3f862825330c1b39102771f2227b225389301 Aug 04 21:30:00.367519 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.367290 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv"] Aug 04 21:30:00.369687 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:30:00.369655 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d4b193_9c95_468f_ba41_a1fe2068b22d.slice/crio-482a2e5b4aa9ed0a943eefe7a7218526d4df192034d46b12405ecdfe5de1a264 WatchSource:0}: Error finding container 482a2e5b4aa9ed0a943eefe7a7218526d4df192034d46b12405ecdfe5de1a264: Status 404 returned error can't find the container with id 482a2e5b4aa9ed0a943eefe7a7218526d4df192034d46b12405ecdfe5de1a264 Aug 04 21:30:00.395732 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.395704 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p"] Aug 04 21:30:00.398174 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:30:00.398145 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeef7bf3_e059_4161_9f63_2dcc6f78a985.slice/crio-e49029645a3803db305da2cad57c149549e644f859d7836e079d5a309ad451f0 WatchSource:0}: Error finding container e49029645a3803db305da2cad57c149549e644f859d7836e079d5a309ad451f0: Status 404 returned error can't find the container with id e49029645a3803db305da2cad57c149549e644f859d7836e079d5a309ad451f0 Aug 04 21:30:00.690404 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.690360 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc" event={"ID":"b02c9e4e-e482-4810-92e6-8f481d04a077","Type":"ContainerStarted","Data":"aec11bf7aa62b7ec8ab9520b7a646aa872832c22d1910172463cef898b2f0bce"} Aug 04 21:30:00.690607 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.690424 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc" event={"ID":"b02c9e4e-e482-4810-92e6-8f481d04a077","Type":"ContainerStarted","Data":"deec87834888193427673827c3c3f862825330c1b39102771f2227b225389301"} Aug 04 21:30:00.691975 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.691935 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv" event={"ID":"99d4b193-9c95-468f-ba41-a1fe2068b22d","Type":"ContainerStarted","Data":"e5006406b0f47cef3533335e43e0e303939644768c9c88cf7ce0529be567c485"} Aug 04 21:30:00.692138 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.691993 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv" event={"ID":"99d4b193-9c95-468f-ba41-a1fe2068b22d","Type":"ContainerStarted","Data":"482a2e5b4aa9ed0a943eefe7a7218526d4df192034d46b12405ecdfe5de1a264"} Aug 04 21:30:00.693109 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.693084 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p" event={"ID":"aeef7bf3-e059-4161-9f63-2dcc6f78a985","Type":"ContainerStarted","Data":"e49029645a3803db305da2cad57c149549e644f859d7836e079d5a309ad451f0"} Aug 04 21:30:00.709860 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.709797 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc" podStartSLOduration=0.709780341 podStartE2EDuration="709.780341ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:30:00.708424358 +0000 UTC m=+2934.855294936" watchObservedRunningTime="2026-08-04 21:30:00.709780341 +0000 UTC m=+2934.856650895" Aug 04 21:30:00.726021 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:00.725895 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv" podStartSLOduration=0.725876537 podStartE2EDuration="725.876537ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:30:00.724545691 +0000 UTC m=+2934.871416248" watchObservedRunningTime="2026-08-04 21:30:00.725876537 +0000 UTC m=+2934.872747120" Aug 04 21:30:01.699809 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:01.699776 2576 generic.go:356] "Generic (PLEG): container finished" podID="aeef7bf3-e059-4161-9f63-2dcc6f78a985" containerID="1704c5d04f98ad50f1ebf90c5b282857e5987bd3e7f8ff6837e1e2decaa32e8e" exitCode=0 Aug 04 21:30:01.700284 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:01.699866 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p" event={"ID":"aeef7bf3-e059-4161-9f63-2dcc6f78a985","Type":"ContainerDied","Data":"1704c5d04f98ad50f1ebf90c5b282857e5987bd3e7f8ff6837e1e2decaa32e8e"} Aug 04 21:30:01.701335 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:01.701309 2576 generic.go:356] "Generic (PLEG): container finished" podID="b02c9e4e-e482-4810-92e6-8f481d04a077" containerID="aec11bf7aa62b7ec8ab9520b7a646aa872832c22d1910172463cef898b2f0bce" exitCode=0 Aug 04 21:30:01.701468 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:01.701339 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc" event={"ID":"b02c9e4e-e482-4810-92e6-8f481d04a077","Type":"ContainerDied","Data":"aec11bf7aa62b7ec8ab9520b7a646aa872832c22d1910172463cef898b2f0bce"} Aug 04 21:30:01.702760 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:01.702742 2576 generic.go:356] "Generic (PLEG): container finished" podID="99d4b193-9c95-468f-ba41-a1fe2068b22d" containerID="e5006406b0f47cef3533335e43e0e303939644768c9c88cf7ce0529be567c485" exitCode=0 Aug 04 21:30:01.702819 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:01.702807 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv" event={"ID":"99d4b193-9c95-468f-ba41-a1fe2068b22d","Type":"ContainerDied","Data":"e5006406b0f47cef3533335e43e0e303939644768c9c88cf7ce0529be567c485"} Aug 04 21:30:02.863155 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:02.859825 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p" Aug 04 21:30:02.904222 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:02.904184 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc" Aug 04 21:30:02.908020 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:02.907995 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv" Aug 04 21:30:03.713868 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:03.713835 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc" Aug 04 21:30:03.713868 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:03.713846 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764650-5bpcc" event={"ID":"b02c9e4e-e482-4810-92e6-8f481d04a077","Type":"ContainerDied","Data":"deec87834888193427673827c3c3f862825330c1b39102771f2227b225389301"} Aug 04 21:30:03.714149 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:03.713899 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="deec87834888193427673827c3c3f862825330c1b39102771f2227b225389301" Aug 04 21:30:03.715180 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:03.715161 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv" Aug 04 21:30:03.715301 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:03.715230 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv" event={"ID":"99d4b193-9c95-468f-ba41-a1fe2068b22d","Type":"ContainerDied","Data":"482a2e5b4aa9ed0a943eefe7a7218526d4df192034d46b12405ecdfe5de1a264"} Aug 04 21:30:03.715360 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:03.715320 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="482a2e5b4aa9ed0a943eefe7a7218526d4df192034d46b12405ecdfe5de1a264" Aug 04 21:30:03.716585 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:03.716564 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p" event={"ID":"aeef7bf3-e059-4161-9f63-2dcc6f78a985","Type":"ContainerDied","Data":"e49029645a3803db305da2cad57c149549e644f859d7836e079d5a309ad451f0"} Aug 04 21:30:03.716704 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:03.716605 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e49029645a3803db305da2cad57c149549e644f859d7836e079d5a309ad451f0" Aug 04 21:30:03.716704 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:03.716573 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p" Aug 04 21:30:25.505510 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:25.505464 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:30:59.450141 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:30:59.450098 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:31:00.297980 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:31:00.297942 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:31:06.710866 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:31:06.710833 2576 scope.go:117] "RemoveContainer" containerID="b32d651be6c473a47c340cf36615f18b2b4123e4e81fb3ced70f0ea6d3937fe3" Aug 04 21:31:11.063192 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:31:11.063152 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:31:42.288111 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:31:42.288071 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:32:56.434204 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:32:56.434164 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:32:57.287380 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:32:57.287341 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:33:10.254585 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:33:10.254537 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:33:28.902396 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:33:28.902317 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:34:39.671521 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:34:39.671482 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:34:40.468233 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:34:40.468202 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:00.215280 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.215183 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:00.395108 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.395069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:00.495798 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.495700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:00.503266 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503226 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8"] Aug 04 21:35:00.503599 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503584 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b02c9e4e-e482-4810-92e6-8f481d04a077" containerName="cleanup" Aug 04 21:35:00.503599 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503600 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b02c9e4e-e482-4810-92e6-8f481d04a077" containerName="cleanup" Aug 04 21:35:00.503699 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503614 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d4b193-9c95-468f-ba41-a1fe2068b22d" containerName="cleanup" Aug 04 21:35:00.503699 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503620 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d4b193-9c95-468f-ba41-a1fe2068b22d" containerName="cleanup" Aug 04 21:35:00.503699 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503626 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeef7bf3-e059-4161-9f63-2dcc6f78a985" containerName="cleanup" Aug 04 21:35:00.503699 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503632 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeef7bf3-e059-4161-9f63-2dcc6f78a985" containerName="cleanup" Aug 04 21:35:00.503699 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503691 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d4b193-9c95-468f-ba41-a1fe2068b22d" containerName="cleanup" Aug 04 21:35:00.503699 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503699 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeef7bf3-e059-4161-9f63-2dcc6f78a985" containerName="cleanup" Aug 04 21:35:00.503885 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.503709 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b02c9e4e-e482-4810-92e6-8f481d04a077" containerName="cleanup" Aug 04 21:35:00.506777 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.506758 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.509392 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.509371 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-e783f6-openshift-default-dockercfg-jdqb4\"" Aug 04 21:35:00.521512 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.521480 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8"] Aug 04 21:35:00.590994 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.590948 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-data\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.591219 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.591055 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64k4b\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-kube-api-access-64k4b\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.591219 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.591093 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-socket\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.591219 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.591119 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-podinfo\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.591219 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.591145 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-token\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.591453 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.591242 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-credential-socket\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.591453 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.591315 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-envoy\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.591453 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.591345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-certs\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.591453 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.591390 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/201ca0b4-08df-4e24-9bce-9f458291c54e-istiod-ca-cert\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.625679 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.625589 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:00.692052 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-envoy\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692052 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692057 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-certs\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692310 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692086 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/201ca0b4-08df-4e24-9bce-9f458291c54e-istiod-ca-cert\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692310 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692106 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-data\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692310 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692145 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64k4b\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-kube-api-access-64k4b\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692310 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692169 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-socket\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692310 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692194 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-podinfo\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692310 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692217 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-token\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692310 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692285 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-credential-socket\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692686 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692498 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-certs\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692686 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692627 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-data\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692686 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692654 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-socket\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.692832 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.692746 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-credential-socket\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.693062 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.693041 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/201ca0b4-08df-4e24-9bce-9f458291c54e-istiod-ca-cert\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.694687 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.694658 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-envoy\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.694934 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.694897 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-podinfo\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.701923 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.701851 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64k4b\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-kube-api-access-64k4b\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.702224 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.702203 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-token\") pod \"e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.817425 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.817332 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:00.887491 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.887453 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:00.970254 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.970225 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8"] Aug 04 21:35:00.972202 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:35:00.972168 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201ca0b4_08df_4e24_9bce_9f458291c54e.slice/crio-ae0451e77d320b1f31d204316dbe009268239325406744e93c97ae2866612cea WatchSource:0}: Error finding container ae0451e77d320b1f31d204316dbe009268239325406744e93c97ae2866612cea: Status 404 returned error can't find the container with id ae0451e77d320b1f31d204316dbe009268239325406744e93c97ae2866612cea Aug 04 21:35:00.974369 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.974336 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:00.974475 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.974416 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:00.974475 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:00.974466 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:01.118845 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:01.118801 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" event={"ID":"201ca0b4-08df-4e24-9bce-9f458291c54e","Type":"ContainerStarted","Data":"d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5"} Aug 04 21:35:01.119048 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:01.118873 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" event={"ID":"201ca0b4-08df-4e24-9bce-9f458291c54e","Type":"ContainerStarted","Data":"ae0451e77d320b1f31d204316dbe009268239325406744e93c97ae2866612cea"} Aug 04 21:35:01.151001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:01.150932 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" podStartSLOduration=1.150890575 podStartE2EDuration="1.150890575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:35:01.149279471 +0000 UTC m=+3235.296150027" watchObservedRunningTime="2026-08-04 21:35:01.150890575 +0000 UTC m=+3235.297761132" Aug 04 21:35:01.465135 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:01.465036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:01.817520 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:01.817427 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:02.823406 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:02.823375 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:03.122428 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:03.122396 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:03.129385 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:03.129357 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:03.130388 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:03.130369 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:03.230005 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:03.229971 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:04.237467 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:04.237430 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:04.481489 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:04.481455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:04.852180 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:04.852147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:04.952434 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:04.952399 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:17.199422 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:17.199383 2576 generic.go:356] "Generic (PLEG): container finished" podID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerID="d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5" exitCode=0 Aug 04 21:35:17.199868 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:17.199455 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" event={"ID":"201ca0b4-08df-4e24-9bce-9f458291c54e","Type":"ContainerDied","Data":"d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5"} Aug 04 21:35:17.199868 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:17.199853 2576 scope.go:117] "RemoveContainer" containerID="d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5" Aug 04 21:35:17.200920 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:17.200866 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:17.201241 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:17.200993 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:17.201241 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:17.201030 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:18.129449 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:18.129378 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:18.205710 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:18.205665 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" event={"ID":"201ca0b4-08df-4e24-9bce-9f458291c54e","Type":"ContainerStarted","Data":"45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3"} Aug 04 21:35:18.818268 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:18.818224 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:18.819407 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:18.819379 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Aug 04 21:35:18.819536 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:18.819431 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Aug 04 21:35:19.818553 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:19.818511 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Aug 04 21:35:19.818984 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:19.818567 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Aug 04 21:35:20.818605 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:20.818560 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Aug 04 21:35:20.819142 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:20.818628 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Aug 04 21:35:21.818205 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:21.818166 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Aug 04 21:35:21.818400 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:21.818235 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Aug 04 21:35:22.818422 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:22.818376 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Aug 04 21:35:22.818818 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:22.818438 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Aug 04 21:35:23.822682 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:23.822652 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:23.823065 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:23.822921 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:23.823607 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:23.823590 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:35:28.118976 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:28.118940 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:28.827692 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:28.827640 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:29.102728 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.102694 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b"] Aug 04 21:35:29.105379 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.105356 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.108188 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.108167 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-80be56-openshift-default-dockercfg-mdqwt\"" Aug 04 21:35:29.124296 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.124254 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b"] Aug 04 21:35:29.166634 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.166594 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-envoy\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.166634 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.166637 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-certs\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.166938 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.166655 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl8sh\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-kube-api-access-pl8sh\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.166938 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.166777 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b890df3-e601-4ea0-9640-fc93cdefc442-istiod-ca-cert\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.166938 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.166825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-credential-socket\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.166938 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.166857 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-podinfo\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.167113 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.166978 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-socket\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.167113 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.167038 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-data\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.167113 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.167065 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-token\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.170211 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.170178 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:29.267555 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.267519 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-socket\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.267729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.267581 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-data\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.267729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.267600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-token\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.267729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.267653 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-envoy\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.267729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.267674 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-certs\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.267729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.267689 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl8sh\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-kube-api-access-pl8sh\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.267729 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.267729 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b890df3-e601-4ea0-9640-fc93cdefc442-istiod-ca-cert\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.268062 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.267753 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-credential-socket\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.268062 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.267816 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-podinfo\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.268241 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.268083 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-certs\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.268241 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.268108 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-data\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.268241 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.268212 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-socket\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.268241 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.268229 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-credential-socket\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.268570 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.268495 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b890df3-e601-4ea0-9640-fc93cdefc442-istiod-ca-cert\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.270213 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.270191 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-envoy\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.270547 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.270525 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-podinfo\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.280774 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.280752 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-token\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.281239 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.281216 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl8sh\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-kube-api-access-pl8sh\") pod \"e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.415634 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.415546 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:29.442568 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.442536 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:29.562228 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.562197 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b"] Aug 04 21:35:29.564737 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:35:29.564711 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b890df3_e601_4ea0_9640_fc93cdefc442.slice/crio-061cd1af6f1dc39fee6a745b1d28b86c262bc8d55a472c3a9c3d22a149ed3a70 WatchSource:0}: Error finding container 061cd1af6f1dc39fee6a745b1d28b86c262bc8d55a472c3a9c3d22a149ed3a70: Status 404 returned error can't find the container with id 061cd1af6f1dc39fee6a745b1d28b86c262bc8d55a472c3a9c3d22a149ed3a70 Aug 04 21:35:29.566982 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.566951 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:29.567043 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.567023 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:29.567084 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:29.567052 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:30.267701 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:30.267663 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" event={"ID":"9b890df3-e601-4ea0-9640-fc93cdefc442","Type":"ContainerStarted","Data":"748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1"} Aug 04 21:35:30.268206 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:30.267713 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" event={"ID":"9b890df3-e601-4ea0-9640-fc93cdefc442","Type":"ContainerStarted","Data":"061cd1af6f1dc39fee6a745b1d28b86c262bc8d55a472c3a9c3d22a149ed3a70"} Aug 04 21:35:30.293456 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:30.293400 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" podStartSLOduration=1.293382352 podStartE2EDuration="1.293382352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:35:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:35:30.290509965 +0000 UTC m=+3264.437380519" watchObservedRunningTime="2026-08-04 21:35:30.293382352 +0000 UTC m=+3264.440252908" Aug 04 21:35:30.415979 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:30.415936 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:30.420897 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:30.420868 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:31.273846 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:31.273815 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:31.274845 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:31.274817 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:32.390280 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:32.390249 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:32.491794 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:32.491758 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:33.370784 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:33.370740 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:33.573575 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:33.573541 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:34.001497 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:34.001462 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:34.101956 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:34.101897 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:35:46.275255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:46.275148 2576 patch_prober.go:28] interesting pod/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 21:35:46.275255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:46.275214 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 21:35:46.349167 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:46.349134 2576 generic.go:356] "Generic (PLEG): container finished" podID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerID="748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1" exitCode=0 Aug 04 21:35:46.349353 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:46.349210 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" event={"ID":"9b890df3-e601-4ea0-9640-fc93cdefc442","Type":"ContainerDied","Data":"748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1"} Aug 04 21:35:46.349643 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:46.349626 2576 scope.go:117] "RemoveContainer" containerID="748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1" Aug 04 21:35:46.350730 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:46.350703 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:46.350792 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:46.350779 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:46.350828 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:46.350815 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:35:47.363451 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:47.363417 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" event={"ID":"9b890df3-e601-4ea0-9640-fc93cdefc442","Type":"ContainerStarted","Data":"ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e"} Aug 04 21:35:47.416138 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:47.416102 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:47.416343 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:47.416318 2576 patch_prober.go:28] interesting pod/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 21:35:47.416419 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:47.416397 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 21:35:48.416750 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:48.416717 2576 patch_prober.go:28] interesting pod/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 21:35:48.417178 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:48.416769 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 21:35:49.416368 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:49.416324 2576 patch_prober.go:28] interesting pod/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 21:35:49.416565 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:49.416394 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 21:35:50.416339 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:50.416304 2576 patch_prober.go:28] interesting pod/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 21:35:50.416746 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:50.416358 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 21:35:51.415866 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:51.415831 2576 patch_prober.go:28] interesting pod/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 21:35:51.416078 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:51.415893 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 21:35:52.419799 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:52.419765 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:52.420196 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:52.420139 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:52.420768 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:52.420746 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:35:57.242188 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:35:57.242151 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:36:15.348469 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:15.348429 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:36:54.850969 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:54.850927 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:36:55.188076 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.187981 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz"] Aug 04 21:36:55.191496 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.191472 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.194493 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.194469 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-354c63-openshift-default-dockercfg-mjgrw\"" Aug 04 21:36:55.202770 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.202746 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz"] Aug 04 21:36:55.218313 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.218275 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:36:55.310450 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.310406 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-podinfo\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.310846 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.310821 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64wj4\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-kube-api-access-64wj4\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.311097 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.311067 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9380d60b-a3c3-48a5-8e07-ada91631e447-istiod-ca-cert\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.311264 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.311250 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-certs\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.311416 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.311398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-socket\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.311570 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.311552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-token\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.311773 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.311754 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-envoy\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.311988 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.311971 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-data\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.312140 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.312126 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-credential-socket\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.413512 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.413465 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64wj4\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-kube-api-access-64wj4\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.413731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.413521 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9380d60b-a3c3-48a5-8e07-ada91631e447-istiod-ca-cert\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.413731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.413559 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-certs\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.413731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.413589 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-socket\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.413731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.413611 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-token\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.413731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.413671 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-envoy\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.413731 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.413720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-data\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.414103 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.413749 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-credential-socket\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.414103 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.413824 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-podinfo\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.414103 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.414024 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-certs\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.414264 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.414119 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-socket\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.414264 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.414247 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-data\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.414355 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.414317 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-credential-socket\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.414413 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.414364 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9380d60b-a3c3-48a5-8e07-ada91631e447-istiod-ca-cert\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.416259 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.416240 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-envoy\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.416581 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.416563 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-podinfo\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.422384 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.422339 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-token\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.422571 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.422548 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64wj4\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-kube-api-access-64wj4\") pod \"e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.476656 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.476553 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:36:55.502482 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.502440 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:55.653170 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.653140 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz"] Aug 04 21:36:55.654743 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:36:55.654713 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9380d60b_a3c3_48a5_8e07_ada91631e447.slice/crio-7a23f30ebffdcda30d16516f7bea3931560fd864c52501afa96b9bc224ee529f WatchSource:0}: Error finding container 7a23f30ebffdcda30d16516f7bea3931560fd864c52501afa96b9bc224ee529f: Status 404 returned error can't find the container with id 7a23f30ebffdcda30d16516f7bea3931560fd864c52501afa96b9bc224ee529f Aug 04 21:36:55.657010 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.656971 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:36:55.657121 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.657043 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:36:55.657121 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.657079 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:36:55.698742 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:55.698711 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" event={"ID":"9380d60b-a3c3-48a5-8e07-ada91631e447","Type":"ContainerStarted","Data":"7a23f30ebffdcda30d16516f7bea3931560fd864c52501afa96b9bc224ee529f"} Aug 04 21:36:56.705384 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:56.705348 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" event={"ID":"9380d60b-a3c3-48a5-8e07-ada91631e447","Type":"ContainerStarted","Data":"0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2"} Aug 04 21:36:56.730862 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:56.730811 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" podStartSLOduration=1.730793554 podStartE2EDuration="1.730793554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:36:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:36:56.729616184 +0000 UTC m=+3350.876486740" watchObservedRunningTime="2026-08-04 21:36:56.730793554 +0000 UTC m=+3350.877664109" Aug 04 21:36:57.503420 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:57.503378 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:57.508500 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:57.508473 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:57.710992 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:57.710959 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:57.712029 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:57.712009 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:36:58.679803 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:58.679762 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:36:58.778464 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:36:58.778430 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:00.634955 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:00.634893 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:00.915420 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:00.915318 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:01.090611 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:01.090571 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:01.190614 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:01.190525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:13.390149 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:13.390109 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:15.855276 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:15.855241 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:15.955186 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:15.955148 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:16.811371 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:16.811341 2576 generic.go:356] "Generic (PLEG): container finished" podID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerID="0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2" exitCode=0 Aug 04 21:37:16.811579 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:16.811394 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" event={"ID":"9380d60b-a3c3-48a5-8e07-ada91631e447","Type":"ContainerDied","Data":"0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2"} Aug 04 21:37:16.811804 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:16.811788 2576 scope.go:117] "RemoveContainer" containerID="0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2" Aug 04 21:37:16.812944 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:16.812888 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:37:16.813005 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:16.812975 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:37:16.813040 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:16.813004 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Aug 04 21:37:17.818930 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:17.818873 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" event={"ID":"9380d60b-a3c3-48a5-8e07-ada91631e447","Type":"ContainerStarted","Data":"76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d"} Aug 04 21:37:18.502999 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:18.502955 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:37:18.503298 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:18.503274 2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 04 21:37:18.503350 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:18.503333 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 04 21:37:19.503874 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:19.503828 2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 04 21:37:19.504324 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:19.503896 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 04 21:37:20.503645 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:20.503607 2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 04 21:37:20.503850 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:20.503674 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 04 21:37:21.503069 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:21.503027 2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 04 21:37:21.503484 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:21.503090 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 04 21:37:21.900156 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:21.900122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:22.001056 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:22.001016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:22.101181 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:22.101148 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:22.503262 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:22.503223 2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 04 21:37:22.503646 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:22.503283 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 04 21:37:22.550896 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:22.550861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:23.507024 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:23.506990 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:37:23.507447 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:23.507246 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:37:23.508001 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:23.507984 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:37:29.011644 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:29.011583 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:29.178132 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:29.178090 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:29.278832 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:29.278732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:29.432920 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:29.432870 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz"] Aug 04 21:37:29.433165 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:29.433137 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerName="istio-proxy" containerID="cri-o://76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d" gracePeriod=30 Aug 04 21:37:29.461549 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:29.461511 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:29.725268 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:29.725232 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:34.677593 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.677559 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:37:34.810201 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810098 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9380d60b-a3c3-48a5-8e07-ada91631e447-istiod-ca-cert\") pod \"9380d60b-a3c3-48a5-8e07-ada91631e447\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " Aug 04 21:37:34.810201 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810145 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-data\") pod \"9380d60b-a3c3-48a5-8e07-ada91631e447\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " Aug 04 21:37:34.810201 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810165 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-envoy\") pod \"9380d60b-a3c3-48a5-8e07-ada91631e447\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " Aug 04 21:37:34.810517 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810225 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-credential-socket\") pod \"9380d60b-a3c3-48a5-8e07-ada91631e447\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " Aug 04 21:37:34.810517 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810261 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-podinfo\") pod \"9380d60b-a3c3-48a5-8e07-ada91631e447\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " Aug 04 21:37:34.810517 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810285 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-token\") pod \"9380d60b-a3c3-48a5-8e07-ada91631e447\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " Aug 04 21:37:34.810517 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810334 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64wj4\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-kube-api-access-64wj4\") pod \"9380d60b-a3c3-48a5-8e07-ada91631e447\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " Aug 04 21:37:34.810517 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810420 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-certs\") pod \"9380d60b-a3c3-48a5-8e07-ada91631e447\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " Aug 04 21:37:34.810517 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810455 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-socket\") pod \"9380d60b-a3c3-48a5-8e07-ada91631e447\" (UID: \"9380d60b-a3c3-48a5-8e07-ada91631e447\") " Aug 04 21:37:34.810517 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810465 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-data" (OuterVolumeSpecName: "istio-data") pod "9380d60b-a3c3-48a5-8e07-ada91631e447" (UID: "9380d60b-a3c3-48a5-8e07-ada91631e447"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:37:34.810865 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810507 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9380d60b-a3c3-48a5-8e07-ada91631e447-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9380d60b-a3c3-48a5-8e07-ada91631e447" (UID: "9380d60b-a3c3-48a5-8e07-ada91631e447"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:37:34.810865 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810607 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9380d60b-a3c3-48a5-8e07-ada91631e447" (UID: "9380d60b-a3c3-48a5-8e07-ada91631e447"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:37:34.810865 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810770 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9380d60b-a3c3-48a5-8e07-ada91631e447" (UID: "9380d60b-a3c3-48a5-8e07-ada91631e447"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:37:34.810865 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810802 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9380d60b-a3c3-48a5-8e07-ada91631e447" (UID: "9380d60b-a3c3-48a5-8e07-ada91631e447"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:37:34.810865 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810820 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9380d60b-a3c3-48a5-8e07-ada91631e447-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:37:34.810865 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810838 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:37:34.810865 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.810851 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:37:34.812779 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.812674 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9380d60b-a3c3-48a5-8e07-ada91631e447" (UID: "9380d60b-a3c3-48a5-8e07-ada91631e447"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:37:34.812779 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.812673 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9380d60b-a3c3-48a5-8e07-ada91631e447" (UID: "9380d60b-a3c3-48a5-8e07-ada91631e447"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:37:34.812779 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.812756 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-kube-api-access-64wj4" (OuterVolumeSpecName: "kube-api-access-64wj4") pod "9380d60b-a3c3-48a5-8e07-ada91631e447" (UID: "9380d60b-a3c3-48a5-8e07-ada91631e447"). InnerVolumeSpecName "kube-api-access-64wj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:37:34.812779 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.812763 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-token" (OuterVolumeSpecName: "istio-token") pod "9380d60b-a3c3-48a5-8e07-ada91631e447" (UID: "9380d60b-a3c3-48a5-8e07-ada91631e447"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:37:34.911523 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.911492 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:37:34.911523 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.911519 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:37:34.911523 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.911528 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64wj4\" (UniqueName: \"kubernetes.io/projected/9380d60b-a3c3-48a5-8e07-ada91631e447-kube-api-access-64wj4\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:37:34.911844 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.911539 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:37:34.911844 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.911548 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:37:34.911844 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.911555 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9380d60b-a3c3-48a5-8e07-ada91631e447-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:37:34.916648 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.916622 2576 generic.go:356] "Generic (PLEG): container finished" podID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerID="76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d" exitCode=0 Aug 04 21:37:34.916778 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.916671 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" Aug 04 21:37:34.916778 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.916711 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" event={"ID":"9380d60b-a3c3-48a5-8e07-ada91631e447","Type":"ContainerDied","Data":"76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d"} Aug 04 21:37:34.916778 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.916766 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz" event={"ID":"9380d60b-a3c3-48a5-8e07-ada91631e447","Type":"ContainerDied","Data":"7a23f30ebffdcda30d16516f7bea3931560fd864c52501afa96b9bc224ee529f"} Aug 04 21:37:34.916878 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.916790 2576 scope.go:117] "RemoveContainer" containerID="76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d" Aug 04 21:37:34.926888 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.926872 2576 scope.go:117] "RemoveContainer" containerID="0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2" Aug 04 21:37:34.934842 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.934799 2576 scope.go:117] "RemoveContainer" containerID="76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d" Aug 04 21:37:34.935143 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:37:34.935122 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d\": container with ID starting with 76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d not found: ID does not exist" containerID="76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d" Aug 04 21:37:34.935203 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.935152 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d"} err="failed to get container status \"76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d\": rpc error: code = NotFound desc = could not find container \"76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d\": container with ID starting with 76b6ae5b287bfe9411ab6017d6ab2f415b5affd8516715c9cf9804cdac104b3d not found: ID does not exist" Aug 04 21:37:34.935203 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.935169 2576 scope.go:117] "RemoveContainer" containerID="0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2" Aug 04 21:37:34.935411 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:37:34.935389 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2\": container with ID starting with 0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2 not found: ID does not exist" containerID="0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2" Aug 04 21:37:34.935469 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.935421 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2"} err="failed to get container status \"0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2\": rpc error: code = NotFound desc = could not find container \"0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2\": container with ID starting with 0d140d028c701cb1d80a20c0a3d7e15582d539660bdde24e524bd2330f4cd8f2 not found: ID does not exist" Aug 04 21:37:34.942492 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.942467 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz"] Aug 04 21:37:34.946558 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:34.946535 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-354c63-openshift-default-684874b6b4-5tklz"] Aug 04 21:37:36.440068 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:36.440031 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" path="/var/lib/kubelet/pods/9380d60b-a3c3-48a5-8e07-ada91631e447/volumes" Aug 04 21:37:51.822889 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:51.822854 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:51.920038 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:51.920001 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:52.016464 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:52.016424 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:52.089567 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:52.089477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:52.399946 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:52.399851 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:37:57.471222 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:37:57.471131 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:33.437805 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:33.437758 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:35.058362 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:35.058324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:35.586278 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:35.586233 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:41.839753 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:41.839711 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:41.963318 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:41.963279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:42.160572 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:42.160539 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8"] Aug 04 21:38:42.160772 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:42.160751 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" containerID="cri-o://45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3" gracePeriod=30 Aug 04 21:38:42.175136 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:42.175107 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:42.426361 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:42.426272 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:47.412755 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.412722 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:38:47.509787 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.509712 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-data\") pod \"201ca0b4-08df-4e24-9bce-9f458291c54e\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " Aug 04 21:38:47.509787 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.509751 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-certs\") pod \"201ca0b4-08df-4e24-9bce-9f458291c54e\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " Aug 04 21:38:47.509787 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.509780 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-socket\") pod \"201ca0b4-08df-4e24-9bce-9f458291c54e\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " Aug 04 21:38:47.510110 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.509812 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/201ca0b4-08df-4e24-9bce-9f458291c54e-istiod-ca-cert\") pod \"201ca0b4-08df-4e24-9bce-9f458291c54e\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " Aug 04 21:38:47.510110 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.509855 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-podinfo\") pod \"201ca0b4-08df-4e24-9bce-9f458291c54e\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " Aug 04 21:38:47.510110 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.509933 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-envoy\") pod \"201ca0b4-08df-4e24-9bce-9f458291c54e\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " Aug 04 21:38:47.510110 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.509970 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64k4b\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-kube-api-access-64k4b\") pod \"201ca0b4-08df-4e24-9bce-9f458291c54e\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " Aug 04 21:38:47.510110 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.510013 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-token\") pod \"201ca0b4-08df-4e24-9bce-9f458291c54e\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " Aug 04 21:38:47.510110 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.510088 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-data" (OuterVolumeSpecName: "istio-data") pod "201ca0b4-08df-4e24-9bce-9f458291c54e" (UID: "201ca0b4-08df-4e24-9bce-9f458291c54e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:38:47.510110 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.510083 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "201ca0b4-08df-4e24-9bce-9f458291c54e" (UID: "201ca0b4-08df-4e24-9bce-9f458291c54e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:38:47.510455 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.510125 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-credential-socket\") pod \"201ca0b4-08df-4e24-9bce-9f458291c54e\" (UID: \"201ca0b4-08df-4e24-9bce-9f458291c54e\") " Aug 04 21:38:47.510455 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.510426 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:38:47.510455 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.510440 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:38:47.510455 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.510117 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "201ca0b4-08df-4e24-9bce-9f458291c54e" (UID: "201ca0b4-08df-4e24-9bce-9f458291c54e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:38:47.510660 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.510370 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "201ca0b4-08df-4e24-9bce-9f458291c54e" (UID: "201ca0b4-08df-4e24-9bce-9f458291c54e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:38:47.510660 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.510367 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/201ca0b4-08df-4e24-9bce-9f458291c54e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "201ca0b4-08df-4e24-9bce-9f458291c54e" (UID: "201ca0b4-08df-4e24-9bce-9f458291c54e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:38:47.512247 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.512212 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "201ca0b4-08df-4e24-9bce-9f458291c54e" (UID: "201ca0b4-08df-4e24-9bce-9f458291c54e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:38:47.512362 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.512316 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "201ca0b4-08df-4e24-9bce-9f458291c54e" (UID: "201ca0b4-08df-4e24-9bce-9f458291c54e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:38:47.512723 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.512701 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-token" (OuterVolumeSpecName: "istio-token") pod "201ca0b4-08df-4e24-9bce-9f458291c54e" (UID: "201ca0b4-08df-4e24-9bce-9f458291c54e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:38:47.512842 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.512718 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-kube-api-access-64k4b" (OuterVolumeSpecName: "kube-api-access-64k4b") pod "201ca0b4-08df-4e24-9bce-9f458291c54e" (UID: "201ca0b4-08df-4e24-9bce-9f458291c54e"). InnerVolumeSpecName "kube-api-access-64k4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:38:47.611647 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.611614 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:38:47.611647 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.611644 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64k4b\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-kube-api-access-64k4b\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:38:47.611647 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.611654 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:38:47.611866 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.611663 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:38:47.611866 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.611671 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/201ca0b4-08df-4e24-9bce-9f458291c54e-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:38:47.611866 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.611679 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/201ca0b4-08df-4e24-9bce-9f458291c54e-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:38:47.611866 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:47.611687 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/201ca0b4-08df-4e24-9bce-9f458291c54e-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:38:48.269080 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.269049 2576 generic.go:356] "Generic (PLEG): container finished" podID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerID="45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3" exitCode=0 Aug 04 21:38:48.269255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.269102 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" Aug 04 21:38:48.269255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.269138 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" event={"ID":"201ca0b4-08df-4e24-9bce-9f458291c54e","Type":"ContainerDied","Data":"45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3"} Aug 04 21:38:48.269255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.269197 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8" event={"ID":"201ca0b4-08df-4e24-9bce-9f458291c54e","Type":"ContainerDied","Data":"ae0451e77d320b1f31d204316dbe009268239325406744e93c97ae2866612cea"} Aug 04 21:38:48.269255 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.269231 2576 scope.go:117] "RemoveContainer" containerID="45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3" Aug 04 21:38:48.278570 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.278447 2576 scope.go:117] "RemoveContainer" containerID="d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5" Aug 04 21:38:48.286689 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.286671 2576 scope.go:117] "RemoveContainer" containerID="45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3" Aug 04 21:38:48.286997 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:38:48.286977 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3\": container with ID starting with 45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3 not found: ID does not exist" containerID="45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3" Aug 04 21:38:48.287058 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.287005 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3"} err="failed to get container status \"45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3\": rpc error: code = NotFound desc = could not find container \"45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3\": container with ID starting with 45ffc29b93863ee2dda2ee3d58f6ca69bd887d42168eb05541a79344c18a80e3 not found: ID does not exist" Aug 04 21:38:48.287058 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.287024 2576 scope.go:117] "RemoveContainer" containerID="d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5" Aug 04 21:38:48.287253 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:38:48.287235 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5\": container with ID starting with d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5 not found: ID does not exist" containerID="d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5" Aug 04 21:38:48.287304 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.287260 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5"} err="failed to get container status \"d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5\": rpc error: code = NotFound desc = could not find container \"d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5\": container with ID starting with d48ddc18cab1fe135e30dd2acb7ff9b103e4f19a58f843e2882d8a5c4838c9d5 not found: ID does not exist" Aug 04 21:38:48.292391 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.292365 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8"] Aug 04 21:38:48.297744 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.297719 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-e783f6-openshift-default-69568ff684-db5h8"] Aug 04 21:38:48.440686 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.440654 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" path="/var/lib/kubelet/pods/201ca0b4-08df-4e24-9bce-9f458291c54e/volumes" Aug 04 21:38:48.868347 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.868312 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:48.968081 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:48.968037 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:49.044590 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:49.044560 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:49.122439 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:49.122269 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:49.558134 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:49.558035 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:56.517131 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:56.517093 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:56.618728 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:56.618683 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:56.762988 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:56.762956 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b"] Aug 04 21:38:56.763189 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:56.763168 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" containerID="cri-o://ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e" gracePeriod=30 Aug 04 21:38:56.798226 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:56.798125 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:38:57.029197 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:38:57.029148 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:02.008091 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.008060 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:39:02.154198 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154156 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-envoy\") pod \"9b890df3-e601-4ea0-9640-fc93cdefc442\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " Aug 04 21:39:02.154365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154229 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-credential-socket\") pod \"9b890df3-e601-4ea0-9640-fc93cdefc442\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " Aug 04 21:39:02.154365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154257 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-token\") pod \"9b890df3-e601-4ea0-9640-fc93cdefc442\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " Aug 04 21:39:02.154365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154273 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-data\") pod \"9b890df3-e601-4ea0-9640-fc93cdefc442\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " Aug 04 21:39:02.154365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154297 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-socket\") pod \"9b890df3-e601-4ea0-9640-fc93cdefc442\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " Aug 04 21:39:02.154365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154323 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-certs\") pod \"9b890df3-e601-4ea0-9640-fc93cdefc442\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " Aug 04 21:39:02.154635 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154372 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl8sh\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-kube-api-access-pl8sh\") pod \"9b890df3-e601-4ea0-9640-fc93cdefc442\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " Aug 04 21:39:02.154635 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154408 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b890df3-e601-4ea0-9640-fc93cdefc442-istiod-ca-cert\") pod \"9b890df3-e601-4ea0-9640-fc93cdefc442\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " Aug 04 21:39:02.154635 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154437 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-podinfo\") pod \"9b890df3-e601-4ea0-9640-fc93cdefc442\" (UID: \"9b890df3-e601-4ea0-9640-fc93cdefc442\") " Aug 04 21:39:02.154635 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154593 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b890df3-e601-4ea0-9640-fc93cdefc442" (UID: "9b890df3-e601-4ea0-9640-fc93cdefc442"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:02.154635 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154606 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b890df3-e601-4ea0-9640-fc93cdefc442" (UID: "9b890df3-e601-4ea0-9640-fc93cdefc442"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:02.154851 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154617 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b890df3-e601-4ea0-9640-fc93cdefc442" (UID: "9b890df3-e601-4ea0-9640-fc93cdefc442"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:02.154851 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154666 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b890df3-e601-4ea0-9640-fc93cdefc442" (UID: "9b890df3-e601-4ea0-9640-fc93cdefc442"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:02.154986 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154863 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:02.154986 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154887 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:02.155100 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.154931 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b890df3-e601-4ea0-9640-fc93cdefc442-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b890df3-e601-4ea0-9640-fc93cdefc442" (UID: "9b890df3-e601-4ea0-9640-fc93cdefc442"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:39:02.155100 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.155040 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:02.155208 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.155110 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:02.156812 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.156782 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b890df3-e601-4ea0-9640-fc93cdefc442" (UID: "9b890df3-e601-4ea0-9640-fc93cdefc442"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:39:02.156812 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.156787 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b890df3-e601-4ea0-9640-fc93cdefc442" (UID: "9b890df3-e601-4ea0-9640-fc93cdefc442"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:02.156812 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.156789 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b890df3-e601-4ea0-9640-fc93cdefc442" (UID: "9b890df3-e601-4ea0-9640-fc93cdefc442"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:39:02.157019 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.156883 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-kube-api-access-pl8sh" (OuterVolumeSpecName: "kube-api-access-pl8sh") pod "9b890df3-e601-4ea0-9640-fc93cdefc442" (UID: "9b890df3-e601-4ea0-9640-fc93cdefc442"). InnerVolumeSpecName "kube-api-access-pl8sh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:39:02.255810 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.255759 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:02.255810 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.255803 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl8sh\" (UniqueName: \"kubernetes.io/projected/9b890df3-e601-4ea0-9640-fc93cdefc442-kube-api-access-pl8sh\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:02.255810 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.255815 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b890df3-e601-4ea0-9640-fc93cdefc442-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:02.256070 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.255826 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:02.256070 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.255846 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b890df3-e601-4ea0-9640-fc93cdefc442-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:02.340879 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.340842 2576 generic.go:356] "Generic (PLEG): container finished" podID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerID="ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e" exitCode=0 Aug 04 21:39:02.341077 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.340890 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" Aug 04 21:39:02.341077 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.340931 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" event={"ID":"9b890df3-e601-4ea0-9640-fc93cdefc442","Type":"ContainerDied","Data":"ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e"} Aug 04 21:39:02.341077 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.340982 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b" event={"ID":"9b890df3-e601-4ea0-9640-fc93cdefc442","Type":"ContainerDied","Data":"061cd1af6f1dc39fee6a745b1d28b86c262bc8d55a472c3a9c3d22a149ed3a70"} Aug 04 21:39:02.341077 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.341005 2576 scope.go:117] "RemoveContainer" containerID="ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e" Aug 04 21:39:02.350172 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.350153 2576 scope.go:117] "RemoveContainer" containerID="748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1" Aug 04 21:39:02.358623 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.358606 2576 scope.go:117] "RemoveContainer" containerID="ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e" Aug 04 21:39:02.358861 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:39:02.358841 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e\": container with ID starting with ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e not found: ID does not exist" containerID="ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e" Aug 04 21:39:02.358945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.358870 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e"} err="failed to get container status \"ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e\": rpc error: code = NotFound desc = could not find container \"ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e\": container with ID starting with ac3c86287769192dfb2ec4f19ea39f9feecc17cfcc87ec99d0d0da4caa22015e not found: ID does not exist" Aug 04 21:39:02.358945 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.358888 2576 scope.go:117] "RemoveContainer" containerID="748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1" Aug 04 21:39:02.359121 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:39:02.359106 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1\": container with ID starting with 748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1 not found: ID does not exist" containerID="748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1" Aug 04 21:39:02.359163 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.359128 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1"} err="failed to get container status \"748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1\": rpc error: code = NotFound desc = could not find container \"748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1\": container with ID starting with 748bf0c5d7224858aee182cd8f6056a01d5f558abd284482b9cc3de789426bc1 not found: ID does not exist" Aug 04 21:39:02.362978 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.362956 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b"] Aug 04 21:39:02.368404 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.368378 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-80be56-openshift-default-7545ccf5dd-h5c4b"] Aug 04 21:39:02.440059 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:02.439977 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" path="/var/lib/kubelet/pods/9b890df3-e601-4ea0-9640-fc93cdefc442/volumes" Aug 04 21:39:03.083289 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:03.083251 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv"] Aug 04 21:39:03.088367 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:03.088340 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a189ec-29764650-7j7dv"] Aug 04 21:39:03.102169 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:03.102134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:03.231402 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:03.231362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:03.466253 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:03.466220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:03.853799 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:03.853737 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:04.440237 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:04.440202 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d4b193-9c95-468f-ba41-a1fe2068b22d" path="/var/lib/kubelet/pods/99d4b193-9c95-468f-ba41-a1fe2068b22d/volumes" Aug 04 21:39:06.784074 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:06.783980 2576 scope.go:117] "RemoveContainer" containerID="e5006406b0f47cef3533335e43e0e303939644768c9c88cf7ce0529be567c485" Aug 04 21:39:11.020583 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:11.020514 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:11.123825 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:11.123789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:11.276300 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:11.276211 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk"] Aug 04 21:39:11.276524 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:11.276476 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" containerID="cri-o://52bbe62fd03716c79bf7942cbe9aa0671b4572605c37f103cc851b9189631013" gracePeriod=30 Aug 04 21:39:11.301401 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:11.301359 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:11.528725 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:11.528617 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:11.858492 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:11.858441 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 21:39:11.858693 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:11.858533 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 21:39:16.413790 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.413755 2576 generic.go:356] "Generic (PLEG): container finished" podID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerID="52bbe62fd03716c79bf7942cbe9aa0671b4572605c37f103cc851b9189631013" exitCode=0 Aug 04 21:39:16.414217 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.413826 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" event={"ID":"f14af029-8b30-4c67-9f2f-9d29cc559943","Type":"ContainerDied","Data":"52bbe62fd03716c79bf7942cbe9aa0671b4572605c37f103cc851b9189631013"} Aug 04 21:39:16.414217 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.413888 2576 scope.go:117] "RemoveContainer" containerID="d59ae457e40a01294d535780052c787d7a58e3ee06a5739116d7172aad88aa0c" Aug 04 21:39:16.526520 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.526493 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:39:16.584358 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584269 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14af029-8b30-4c67-9f2f-9d29cc559943-istiod-ca-cert\") pod \"f14af029-8b30-4c67-9f2f-9d29cc559943\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " Aug 04 21:39:16.584358 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584339 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-data\") pod \"f14af029-8b30-4c67-9f2f-9d29cc559943\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " Aug 04 21:39:16.584358 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584356 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-token\") pod \"f14af029-8b30-4c67-9f2f-9d29cc559943\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " Aug 04 21:39:16.584663 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584375 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-credential-socket\") pod \"f14af029-8b30-4c67-9f2f-9d29cc559943\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " Aug 04 21:39:16.584663 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584391 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-envoy\") pod \"f14af029-8b30-4c67-9f2f-9d29cc559943\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " Aug 04 21:39:16.584663 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584447 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-podinfo\") pod \"f14af029-8b30-4c67-9f2f-9d29cc559943\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " Aug 04 21:39:16.584663 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584493 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwwhz\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-kube-api-access-vwwhz\") pod \"f14af029-8b30-4c67-9f2f-9d29cc559943\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " Aug 04 21:39:16.584663 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584532 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-certs\") pod \"f14af029-8b30-4c67-9f2f-9d29cc559943\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " Aug 04 21:39:16.584663 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584559 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-socket\") pod \"f14af029-8b30-4c67-9f2f-9d29cc559943\" (UID: \"f14af029-8b30-4c67-9f2f-9d29cc559943\") " Aug 04 21:39:16.584981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584689 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-data" (OuterVolumeSpecName: "istio-data") pod "f14af029-8b30-4c67-9f2f-9d29cc559943" (UID: "f14af029-8b30-4c67-9f2f-9d29cc559943"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:16.584981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584785 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f14af029-8b30-4c67-9f2f-9d29cc559943" (UID: "f14af029-8b30-4c67-9f2f-9d29cc559943"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:16.584981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584710 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f14af029-8b30-4c67-9f2f-9d29cc559943-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f14af029-8b30-4c67-9f2f-9d29cc559943" (UID: "f14af029-8b30-4c67-9f2f-9d29cc559943"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:39:16.584981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584897 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14af029-8b30-4c67-9f2f-9d29cc559943-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:16.584981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584949 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:16.584981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584962 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:16.585225 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.584985 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f14af029-8b30-4c67-9f2f-9d29cc559943" (UID: "f14af029-8b30-4c67-9f2f-9d29cc559943"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:16.585342 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.585317 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f14af029-8b30-4c67-9f2f-9d29cc559943" (UID: "f14af029-8b30-4c67-9f2f-9d29cc559943"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:16.586819 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.586793 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f14af029-8b30-4c67-9f2f-9d29cc559943" (UID: "f14af029-8b30-4c67-9f2f-9d29cc559943"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:16.587109 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.587090 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-kube-api-access-vwwhz" (OuterVolumeSpecName: "kube-api-access-vwwhz") pod "f14af029-8b30-4c67-9f2f-9d29cc559943" (UID: "f14af029-8b30-4c67-9f2f-9d29cc559943"). InnerVolumeSpecName "kube-api-access-vwwhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:39:16.587321 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.587294 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f14af029-8b30-4c67-9f2f-9d29cc559943" (UID: "f14af029-8b30-4c67-9f2f-9d29cc559943"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:39:16.587366 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.587327 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-token" (OuterVolumeSpecName: "istio-token") pod "f14af029-8b30-4c67-9f2f-9d29cc559943" (UID: "f14af029-8b30-4c67-9f2f-9d29cc559943"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:39:16.685709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.685669 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:16.685709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.685702 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:16.685709 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.685711 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14af029-8b30-4c67-9f2f-9d29cc559943-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:16.685997 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.685721 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwwhz\" (UniqueName: \"kubernetes.io/projected/f14af029-8b30-4c67-9f2f-9d29cc559943-kube-api-access-vwwhz\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:16.685997 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.685736 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:16.685997 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.685747 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14af029-8b30-4c67-9f2f-9d29cc559943-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:16.952117 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.952080 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p"] Aug 04 21:39:16.954680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.954646 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-cd644a-29764650-zww4p"] Aug 04 21:39:16.955795 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:16.955758 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:17.047887 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:17.047849 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:17.182837 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:17.182762 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:17.420038 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:17.420002 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" Aug 04 21:39:17.420462 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:17.420005 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk" event={"ID":"f14af029-8b30-4c67-9f2f-9d29cc559943","Type":"ContainerDied","Data":"627fafa573d951b9effa7ea1f6dfeeb370a367446306d0c0ac26ba98e31e2c71"} Aug 04 21:39:17.420462 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:17.420169 2576 scope.go:117] "RemoveContainer" containerID="52bbe62fd03716c79bf7942cbe9aa0671b4572605c37f103cc851b9189631013" Aug 04 21:39:17.442687 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:17.442653 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk"] Aug 04 21:39:17.448700 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:17.448675 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a189ec-openshift-default-586749b668-7nlvk"] Aug 04 21:39:17.737591 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:17.737493 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:18.440136 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:18.440092 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeef7bf3-e059-4161-9f63-2dcc6f78a985" path="/var/lib/kubelet/pods/aeef7bf3-e059-4161-9f63-2dcc6f78a985/volumes" Aug 04 21:39:18.440657 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:18.440633 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" path="/var/lib/kubelet/pods/f14af029-8b30-4c67-9f2f-9d29cc559943/volumes" Aug 04 21:39:24.168190 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:24.168144 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:24.244048 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:24.244009 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:24.481374 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:24.481291 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp"] Aug 04 21:39:24.481549 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:24.481506 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" containerID="cri-o://9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130" gracePeriod=30 Aug 04 21:39:24.519607 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:24.519572 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:24.569879 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:24.569842 2576 patch_prober.go:28] interesting pod/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 21:39:24.570042 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:24.569896 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 21:39:24.714795 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:24.714762 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:29.732577 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.732541 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:39:29.801727 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.801642 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5098c4de-c711-488a-a0ee-9a6c7da291b5-istiod-ca-cert\") pod \"5098c4de-c711-488a-a0ee-9a6c7da291b5\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " Aug 04 21:39:29.801727 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.801695 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-podinfo\") pod \"5098c4de-c711-488a-a0ee-9a6c7da291b5\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " Aug 04 21:39:29.801727 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.801726 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-data\") pod \"5098c4de-c711-488a-a0ee-9a6c7da291b5\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " Aug 04 21:39:29.802027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.801754 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-socket\") pod \"5098c4de-c711-488a-a0ee-9a6c7da291b5\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " Aug 04 21:39:29.802027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.801800 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-credential-socket\") pod \"5098c4de-c711-488a-a0ee-9a6c7da291b5\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " Aug 04 21:39:29.802027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.801896 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-certs\") pod \"5098c4de-c711-488a-a0ee-9a6c7da291b5\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " Aug 04 21:39:29.802027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.801960 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr72j\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-kube-api-access-sr72j\") pod \"5098c4de-c711-488a-a0ee-9a6c7da291b5\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " Aug 04 21:39:29.802027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.801991 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-token\") pod \"5098c4de-c711-488a-a0ee-9a6c7da291b5\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " Aug 04 21:39:29.802027 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802025 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-envoy\") pod \"5098c4de-c711-488a-a0ee-9a6c7da291b5\" (UID: \"5098c4de-c711-488a-a0ee-9a6c7da291b5\") " Aug 04 21:39:29.802329 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802108 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-data" (OuterVolumeSpecName: "istio-data") pod "5098c4de-c711-488a-a0ee-9a6c7da291b5" (UID: "5098c4de-c711-488a-a0ee-9a6c7da291b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:29.802329 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802124 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5098c4de-c711-488a-a0ee-9a6c7da291b5" (UID: "5098c4de-c711-488a-a0ee-9a6c7da291b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:29.802329 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802158 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5098c4de-c711-488a-a0ee-9a6c7da291b5" (UID: "5098c4de-c711-488a-a0ee-9a6c7da291b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:29.802329 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802182 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5098c4de-c711-488a-a0ee-9a6c7da291b5" (UID: "5098c4de-c711-488a-a0ee-9a6c7da291b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:29.802329 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802190 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5098c4de-c711-488a-a0ee-9a6c7da291b5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5098c4de-c711-488a-a0ee-9a6c7da291b5" (UID: "5098c4de-c711-488a-a0ee-9a6c7da291b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 21:39:29.802564 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802458 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5098c4de-c711-488a-a0ee-9a6c7da291b5-istiod-ca-cert\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:29.802564 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802479 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-data\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:29.802564 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802492 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:29.802564 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802506 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-credential-socket\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:29.802564 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.802518 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-workload-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:29.804206 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.804181 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5098c4de-c711-488a-a0ee-9a6c7da291b5" (UID: "5098c4de-c711-488a-a0ee-9a6c7da291b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 21:39:29.804496 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.804469 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5098c4de-c711-488a-a0ee-9a6c7da291b5" (UID: "5098c4de-c711-488a-a0ee-9a6c7da291b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 21:39:29.804706 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.804685 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "5098c4de-c711-488a-a0ee-9a6c7da291b5" (UID: "5098c4de-c711-488a-a0ee-9a6c7da291b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:39:29.804706 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.804699 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-kube-api-access-sr72j" (OuterVolumeSpecName: "kube-api-access-sr72j") pod "5098c4de-c711-488a-a0ee-9a6c7da291b5" (UID: "5098c4de-c711-488a-a0ee-9a6c7da291b5"). InnerVolumeSpecName "kube-api-access-sr72j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 21:39:29.903554 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.903521 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr72j\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-kube-api-access-sr72j\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:29.903554 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.903549 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:29.903715 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.903562 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-envoy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:29.903715 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:29.903573 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5098c4de-c711-488a-a0ee-9a6c7da291b5-istio-podinfo\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\"" Aug 04 21:39:30.480405 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.480376 2576 generic.go:356] "Generic (PLEG): container finished" podID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerID="9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130" exitCode=0 Aug 04 21:39:30.480546 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.480437 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" Aug 04 21:39:30.480546 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.480441 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" event={"ID":"5098c4de-c711-488a-a0ee-9a6c7da291b5","Type":"ContainerDied","Data":"9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130"} Aug 04 21:39:30.480637 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.480553 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp" event={"ID":"5098c4de-c711-488a-a0ee-9a6c7da291b5","Type":"ContainerDied","Data":"640e85e7985be5f7020fb785e61674a32970da6d0100b7898a05519e0fc45f65"} Aug 04 21:39:30.480637 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.480574 2576 scope.go:117] "RemoveContainer" containerID="9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130" Aug 04 21:39:30.489314 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.489296 2576 scope.go:117] "RemoveContainer" containerID="34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379" Aug 04 21:39:30.497483 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.497432 2576 scope.go:117] "RemoveContainer" containerID="9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130" Aug 04 21:39:30.497827 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:39:30.497806 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130\": container with ID starting with 9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130 not found: ID does not exist" containerID="9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130" Aug 04 21:39:30.497886 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.497836 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130"} err="failed to get container status \"9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130\": rpc error: code = NotFound desc = could not find container \"9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130\": container with ID starting with 9c3aeddfc4a4ad6992ad1e1b9e40fd1a5f98459ae5a9b62da027addfac7ac130 not found: ID does not exist" Aug 04 21:39:30.497886 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.497853 2576 scope.go:117] "RemoveContainer" containerID="34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379" Aug 04 21:39:30.498233 ip-10-0-139-54 kubenswrapper[2576]: E0804 21:39:30.498212 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379\": container with ID starting with 34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379 not found: ID does not exist" containerID="34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379" Aug 04 21:39:30.498322 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.498236 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379"} err="failed to get container status \"34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379\": rpc error: code = NotFound desc = could not find container \"34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379\": container with ID starting with 34bdd18a52781f29d9ba2d138d2edb6625284ec05f13db4daff48980701d5379 not found: ID does not exist" Aug 04 21:39:30.499028 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.499009 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp"] Aug 04 21:39:30.502514 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:30.502491 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-cd644a-openshift-default-554985d8-2fgtp"] Aug 04 21:39:31.339095 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:31.339060 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6889885b4f-zw2b2_815d611d-3da2-4318-b23d-4816c7633ae1/manager/0.log" Aug 04 21:39:31.346725 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:31.346671 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6889885b4f-zw2b2_815d611d-3da2-4318-b23d-4816c7633ae1/copy-manifests/0.log" Aug 04 21:39:31.920026 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:31.919991 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764635-jhk72_ce242cf8-9969-4de3-8bfe-44e1439e619a/cleanup/0.log" Aug 04 21:39:32.035858 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:32.035829 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764650-5bpcc_b02c9e4e-e482-4810-92e6-8f481d04a077/cleanup/0.log" Aug 04 21:39:32.439382 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:32.439338 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" path="/var/lib/kubelet/pods/5098c4de-c711-488a-a0ee-9a6c7da291b5/volumes" Aug 04 21:39:33.987995 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:33.987959 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc_4aa57cb2-e32d-4c49-a86f-f8dbac7273ce/istio-proxy/0.log" Aug 04 21:39:34.118920 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:34.118873 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rdm86_b3160f39-c643-4471-a95f-bbf6b1c71c0c/discovery/0.log" Aug 04 21:39:34.485320 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:34.485283 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-bbk2x_0cf64b0c-84b7-4d8a-8c4d-a877308ff994/istio-proxy/1.log" Aug 04 21:39:35.181899 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:35.181855 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-2s2xx_aeedafd0-92d3-41e6-b256-a4aa0f3cb1f9/main/0.log" Aug 04 21:39:35.407667 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:35.407637 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8s5lg_b0de5437-789c-4d28-bc0a-1c89def6a0f3/main/0.log" Aug 04 21:39:40.758816 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:40.758778 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:40.857239 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:40.857202 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" limit=1024 Aug 04 21:39:43.708425 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:43.708391 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6j5xp_a43ca81a-b9a5-45da-b3fa-09452b3296f6/global-pull-secret-syncer/0.log" Aug 04 21:39:43.821803 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:43.821761 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ncqlz_eef28aa9-8bf5-4c87-bcb6-245360c85048/konnectivity-agent/0.log" Aug 04 21:39:43.953045 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:43.953016 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-54.ec2.internal_079dd266070347668559bc450b935f9a/haproxy/0.log" Aug 04 21:39:51.280016 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.279934 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x5mp5_b416b2ad-ae78-4a40-a731-5b867e854a9d/node-exporter/0.log" Aug 04 21:39:51.305290 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.305260 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x5mp5_b416b2ad-ae78-4a40-a731-5b867e854a9d/kube-rbac-proxy/0.log" Aug 04 21:39:51.328768 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.328746 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x5mp5_b416b2ad-ae78-4a40-a731-5b867e854a9d/init-textfile/0.log" Aug 04 21:39:51.450007 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.449967 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-7p6bj_401f7af5-f9af-49cd-9d62-106877917d06/kube-rbac-proxy-main/0.log" Aug 04 21:39:51.475980 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.475958 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-7p6bj_401f7af5-f9af-49cd-9d62-106877917d06/kube-rbac-proxy-self/0.log" Aug 04 21:39:51.501581 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.501556 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-7p6bj_401f7af5-f9af-49cd-9d62-106877917d06/openshift-state-metrics/0.log" Aug 04 21:39:51.757231 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.757195 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-7zq46_9f3871a9-49cd-45ef-8c22-940f2fbbe7f3/prometheus-operator/0.log" Aug 04 21:39:51.781365 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.781333 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-7zq46_9f3871a9-49cd-45ef-8c22-940f2fbbe7f3/kube-rbac-proxy/0.log" Aug 04 21:39:51.931557 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.931516 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b78876469-2ppjb_494573a3-8394-4adb-99a8-005042b3de04/thanos-query/0.log" Aug 04 21:39:51.958732 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.958705 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b78876469-2ppjb_494573a3-8394-4adb-99a8-005042b3de04/kube-rbac-proxy-web/0.log" Aug 04 21:39:51.982302 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:51.982275 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b78876469-2ppjb_494573a3-8394-4adb-99a8-005042b3de04/kube-rbac-proxy/0.log" Aug 04 21:39:52.003399 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:52.003368 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b78876469-2ppjb_494573a3-8394-4adb-99a8-005042b3de04/prom-label-proxy/0.log" Aug 04 21:39:52.027234 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:52.027162 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b78876469-2ppjb_494573a3-8394-4adb-99a8-005042b3de04/kube-rbac-proxy-rules/0.log" Aug 04 21:39:52.051277 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:52.051248 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b78876469-2ppjb_494573a3-8394-4adb-99a8-005042b3de04/kube-rbac-proxy-metrics/0.log" Aug 04 21:39:53.963092 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963059 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh"] Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963376 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963387 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963399 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963406 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963413 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963419 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963427 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963432 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963438 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963443 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963453 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963458 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963466 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963472 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963483 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" Aug 04 21:39:53.963488 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963488 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" Aug 04 21:39:53.963981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963538 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" Aug 04 21:39:53.963981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963546 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="201ca0b4-08df-4e24-9bce-9f458291c54e" containerName="istio-proxy" Aug 04 21:39:53.963981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963552 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f14af029-8b30-4c67-9f2f-9d29cc559943" containerName="istio-proxy" Aug 04 21:39:53.963981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963557 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5098c4de-c711-488a-a0ee-9a6c7da291b5" containerName="istio-proxy" Aug 04 21:39:53.963981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963564 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9380d60b-a3c3-48a5-8e07-ada91631e447" containerName="istio-proxy" Aug 04 21:39:53.963981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963570 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" Aug 04 21:39:53.963981 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.963576 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b890df3-e601-4ea0-9640-fc93cdefc442" containerName="istio-proxy" Aug 04 21:39:53.966544 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.966522 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:53.968850 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.968828 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nzqjh\"/\"kube-root-ca.crt\"" Aug 04 21:39:53.968991 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.968828 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nzqjh\"/\"default-dockercfg-cpcnx\"" Aug 04 21:39:53.969668 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.969651 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nzqjh\"/\"openshift-service-ca.crt\"" Aug 04 21:39:53.975868 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:53.975842 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh"] Aug 04 21:39:54.129951 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.129890 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-proc\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.130142 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.129995 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-podres\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.130142 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.130042 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-sys\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.130142 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.130076 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-lib-modules\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.130142 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.130139 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcm9v\" (UniqueName: \"kubernetes.io/projected/bc86c68a-9f93-4810-96b9-3e813b39b9b1-kube-api-access-qcm9v\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.231640 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.231537 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-proc\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.231640 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.231595 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-podres\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.231640 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.231641 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-sys\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.231965 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.231679 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-lib-modules\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.231965 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.231682 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-proc\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.231965 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.231745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcm9v\" (UniqueName: \"kubernetes.io/projected/bc86c68a-9f93-4810-96b9-3e813b39b9b1-kube-api-access-qcm9v\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.231965 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.231785 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-podres\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.231965 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.231748 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-sys\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.231965 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.231814 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc86c68a-9f93-4810-96b9-3e813b39b9b1-lib-modules\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.241004 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.240982 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcm9v\" (UniqueName: \"kubernetes.io/projected/bc86c68a-9f93-4810-96b9-3e813b39b9b1-kube-api-access-qcm9v\") pod \"perf-node-gather-daemonset-xcwbh\" (UID: \"bc86c68a-9f93-4810-96b9-3e813b39b9b1\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.276932 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.276869 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.412560 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.412523 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh"] Aug 04 21:39:54.415843 ip-10-0-139-54 kubenswrapper[2576]: W0804 21:39:54.415813 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbc86c68a_9f93_4810_96b9_3e813b39b9b1.slice/crio-0d6dd60d6c6563cb5aadd16bd46391a1542b2ccdfe1c0790093a6e42738e6ca2 WatchSource:0}: Error finding container 0d6dd60d6c6563cb5aadd16bd46391a1542b2ccdfe1c0790093a6e42738e6ca2: Status 404 returned error can't find the container with id 0d6dd60d6c6563cb5aadd16bd46391a1542b2ccdfe1c0790093a6e42738e6ca2 Aug 04 21:39:54.588680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.588578 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" event={"ID":"bc86c68a-9f93-4810-96b9-3e813b39b9b1","Type":"ContainerStarted","Data":"147109de4bbcf85cdffea17cccfad876ae10609f41a4a5f9e21c95cb0aa113ea"} Aug 04 21:39:54.588680 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.588639 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" event={"ID":"bc86c68a-9f93-4810-96b9-3e813b39b9b1","Type":"ContainerStarted","Data":"0d6dd60d6c6563cb5aadd16bd46391a1542b2ccdfe1c0790093a6e42738e6ca2"} Aug 04 21:39:54.588881 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.588689 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:39:54.609554 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.609497 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" podStartSLOduration=1.6094799850000001 podStartE2EDuration="1.609479985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 21:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:39:54.606844823 +0000 UTC m=+3528.753715377" watchObservedRunningTime="2026-08-04 21:39:54.609479985 +0000 UTC m=+3528.756350540" Aug 04 21:39:54.697504 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:54.697474 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f99ff488f-xsz8h_3cc2fd20-ef93-472e-89a0-d56579654878/console/0.log" Aug 04 21:39:55.448971 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:55.448932 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-9jtjx_76417d59-9d65-441d-92d7-c5f49a592c60/volume-data-source-validator/0.log" Aug 04 21:39:56.556159 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:56.556130 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mn9mw_cd2bb155-816d-4b33-9c0f-528c36d56f55/dns/0.log" Aug 04 21:39:56.578038 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:56.578010 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mn9mw_cd2bb155-816d-4b33-9c0f-528c36d56f55/kube-rbac-proxy/0.log" Aug 04 21:39:56.663198 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:56.663165 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4qgqw_157ecd5c-7c37-4d11-8ec9-947628a5540d/dns-node-resolver/0.log" Aug 04 21:39:57.357194 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:57.357161 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-55kxj_4004db00-a176-456f-88d5-a8204195d9d3/node-ca/0.log" Aug 04 21:39:58.440730 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:58.440695 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc_4aa57cb2-e32d-4c49-a86f-f8dbac7273ce/istio-proxy/0.log" Aug 04 21:39:58.702267 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:58.702182 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rdm86_b3160f39-c643-4471-a95f-bbf6b1c71c0c/discovery/0.log" Aug 04 21:39:58.817784 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:58.817747 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-bbk2x_0cf64b0c-84b7-4d8a-8c4d-a877308ff994/istio-proxy/0.log" Aug 04 21:39:58.830782 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:58.830751 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-bbk2x_0cf64b0c-84b7-4d8a-8c4d-a877308ff994/istio-proxy/1.log" Aug 04 21:39:59.752407 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:39:59.752375 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7cc5c_699b5cd2-1fb4-4fb3-a6c0-1bf61957f8c6/serve-healthcheck-canary/0.log" Aug 04 21:40:00.252766 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:00.252722 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-6slt9_8bf442c4-3348-46ae-ac9c-69b05f9dcbfb/insights-operator/0.log" Aug 04 21:40:00.254604 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:00.254579 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-6slt9_8bf442c4-3348-46ae-ac9c-69b05f9dcbfb/insights-operator/1.log" Aug 04 21:40:00.277104 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:00.277069 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-27h2f_d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c/kube-rbac-proxy/0.log" Aug 04 21:40:00.303500 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:00.303472 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-27h2f_d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c/exporter/0.log" Aug 04 21:40:00.327002 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:00.326964 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-27h2f_d7f8d3e6-cbdf-4e8d-a955-1a1a3489953c/extractor/0.log" Aug 04 21:40:00.606762 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:00.606730 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-xcwbh" Aug 04 21:40:03.560519 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:03.560433 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6889885b4f-zw2b2_815d611d-3da2-4318-b23d-4816c7633ae1/manager/0.log" Aug 04 21:40:03.586884 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:03.586854 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6889885b4f-zw2b2_815d611d-3da2-4318-b23d-4816c7633ae1/copy-manifests/0.log" Aug 04 21:40:04.541611 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:04.541571 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc_4aa57cb2-e32d-4c49-a86f-f8dbac7273ce/istio-proxy/0.log" Aug 04 21:40:04.855745 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:04.855711 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7564c45d7d-zzntw_a884dc58-5aa1-4881-9f01-a41bd5aea854/manager/0.log" Aug 04 21:40:04.874127 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:04.874096 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rdm86_b3160f39-c643-4471-a95f-bbf6b1c71c0c/discovery/0.log" Aug 04 21:40:04.921592 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:04.921556 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-bbk2x_0cf64b0c-84b7-4d8a-8c4d-a877308ff994/istio-proxy/0.log" Aug 04 21:40:04.934217 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:04.934189 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-bbk2x_0cf64b0c-84b7-4d8a-8c4d-a877308ff994/istio-proxy/1.log" Aug 04 21:40:06.432996 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:06.432949 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfccc72kzc_4aa57cb2-e32d-4c49-a86f-f8dbac7273ce/istio-proxy/0.log" Aug 04 21:40:06.759108 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:06.759001 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rdm86_b3160f39-c643-4471-a95f-bbf6b1c71c0c/discovery/0.log" Aug 04 21:40:06.810098 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:06.810065 2576 scope.go:117] "RemoveContainer" containerID="1704c5d04f98ad50f1ebf90c5b282857e5987bd3e7f8ff6837e1e2decaa32e8e" Aug 04 21:40:06.819835 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:06.819797 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-bbk2x_0cf64b0c-84b7-4d8a-8c4d-a877308ff994/istio-proxy/0.log" Aug 04 21:40:06.836191 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:06.836113 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-bbk2x_0cf64b0c-84b7-4d8a-8c4d-a877308ff994/istio-proxy/1.log" Aug 04 21:40:11.599799 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:11.599767 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cbt9z_0aa9cd51-695a-4ad9-b14b-5fe0dc758268/kube-multus-additional-cni-plugins/0.log" Aug 04 21:40:11.624454 ip-10-0-139-54 kubenswrapper[2576]: I0804 21:40:11.624420 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cbt9z_0aa9cd51-695a-4ad9-b14b-5fe0dc758268/egress-router-binary-copy/0.log"