Sep 06 19:09:12.657184 ip-10-0-135-200 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 06 19:09:12.657196 ip-10-0-135-200 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 06 19:09:12.657203 ip-10-0-135-200 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 06 19:09:12.657406 ip-10-0-135-200 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 06 19:09:22.862416 ip-10-0-135-200 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 06 19:09:22.862437 ip-10-0-135-200 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 34187d5f07bc40219841d39beeddb7fe --
Sep 06 19:11:09.695698 ip-10-0-135-200 systemd[1]: Starting Kubernetes Kubelet...
Sep 06 19:11:10.136093 ip-10-0-135-200 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 19:11:10.136093 ip-10-0-135-200 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 06 19:11:10.136093 ip-10-0-135-200 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 19:11:10.136093 ip-10-0-135-200 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 06 19:11:10.136093 ip-10-0-135-200 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 19:11:10.138072 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.137973    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 06 19:11:10.142641 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142623    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 19:11:10.142641 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142642    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142646    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142649    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142652    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142654    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142657    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142660    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142663    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142665    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142668    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142671    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142673    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142676    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142680    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142682    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142685    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142689    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142693    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142695    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142698    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 19:11:10.142714 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142701    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142704    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142706    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142709    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142712    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142715    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142717    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142720    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142723    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142726    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142730    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142733    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142736    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142739    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142742    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142745    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142747    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142756    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142759    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 19:11:10.143192 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142762    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142765    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142767    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142769    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142772    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142774    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142777    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142779    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142781    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142784    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142786    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142788    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142791    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142793    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142796    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142800    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142803    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142806    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142809    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142811    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 19:11:10.143669 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142814    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142816    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142819    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142821    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142824    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142826    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142829    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142833    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142836    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142838    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142841    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142848    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142851    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142853    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142856    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142858    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142861    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142863    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142865    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142868    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 19:11:10.144163 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142870    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142873    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142875    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142878    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142881    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142883    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142887    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142890    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142892    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142895    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142897    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142900    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142903    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142907    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142910    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142912    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142915    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142917    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.142920    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 19:11:10.144681 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144567    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144576    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144579    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144583    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144588    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144591    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144595    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144597    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144600    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144603    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144605    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144608    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144611    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144615    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144618    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144620    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144624    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144626    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144629    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 19:11:10.145139 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144632    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144636    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144640    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144644    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144649    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144652    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144654    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144657    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144659    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144662    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144664    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144667    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144669    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144672    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144674    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144677    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144679    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144682    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144684    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144687    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 19:11:10.145703 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144690    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144693    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144695    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144698    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144701    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144703    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144706    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144708    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144711    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144713    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144715    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144718    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144720    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144723    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144725    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144728    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144730    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144733    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144735    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144738    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 19:11:10.146215 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144740    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144743    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144745    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144748    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144750    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144752    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144755    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144758    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144760    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144763    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144765    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144768    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144771    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144774    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144776    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144779    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144782    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144784    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144786    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144789    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 19:11:10.146831 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144791    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144794    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144796    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144798    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144801    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144804    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144807    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144810    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144812    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144815    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144818    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144820    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144823    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144825    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144827    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144830    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144832    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144834    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144838    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.144840    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 19:11:10.147328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145239    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145243    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145246    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145249    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145252    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145255    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145258    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145260    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145263    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145265    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145270    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145273    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145276    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145279    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145282    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145285    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145287    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145292    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145295    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 19:11:10.147847 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145298    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145300    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145303    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145305    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145308    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145310    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145312    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145315    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145318    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145320    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145322    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145325    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145328    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145330    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145332    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145335    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145337    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145340    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145342    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145345    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 19:11:10.148308 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145347    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145350    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145353    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145355    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145358    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145360    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145363    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145365    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145368    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145370    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145374    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145378    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145381    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145400    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145403    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145406    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145409    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145412    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145414    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 19:11:10.148833 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145417    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145420    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145422    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145425    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145428    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145430    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145433    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145435    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145438    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145454    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145457    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145461    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145464    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145467    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145470    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145473    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145476    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145479    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145481    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145484    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 19:11:10.149297 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145487    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145489    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145492    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145494    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145497    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145499    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145502    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145504    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145507    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145509    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145512    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145514    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145518    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145521    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145523    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145526    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145529    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145532    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145534    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145537    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 19:11:10.149809 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.145539    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145610    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145646    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145652    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145657    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145663    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145668    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145673    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145678    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145681    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145684    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145687    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145691    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145694    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145697    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145699    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145702    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145705    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145709    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145714    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145717    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145721    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145724    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145727    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 06 19:11:10.150294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145731    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145734    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145738    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145741    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145746    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145749    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145751    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145755    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145757    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145761    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145764    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145767    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145770    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145773    2581 flags.go:64] FLAG: --enable-server="true"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145779    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145784    2581 flags.go:64] FLAG: --event-burst="100"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145787    2581 flags.go:64] FLAG: --event-qps="50"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145796    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145801    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145804    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145808    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145811    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145814    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145817    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145820    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 06 19:11:10.150932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145823    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145825    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145828    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145831    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145834    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145837    2581 flags.go:64] FLAG: --feature-gates=""
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145841    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145844    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145847    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145850    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145853    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145857    2581 flags.go:64] FLAG: --help="false"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145860    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145863    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145866    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145869    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145872    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145876    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145879    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145882    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145884    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145888    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145895    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145898    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 06 19:11:10.151604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145901    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145904    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145907    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145910    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145912    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145915    2581 flags.go:64] FLAG: --lock-file=""
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145918    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145921    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145923    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145928    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145931    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145934    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145936    2581 flags.go:64] FLAG: --logging-format="text"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145939    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145942    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145945    2581 flags.go:64] FLAG: --manifest-url=""
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145948    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145952    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145955    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145962    2581 flags.go:64] FLAG: --max-pods="110"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145965    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145968    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145971    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145974    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145976    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 06 19:11:10.152222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145979    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145982    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145989    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145992    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.145997    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146000    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146004    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146010    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146013    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146017    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146020    2581 flags.go:64] FLAG: --port="10250"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146023    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146026    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-039387ae879619c0d"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146029    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146032    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146035    2581 flags.go:64] FLAG: --register-node="true"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146038    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146041    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146044    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146047    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146050    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146053    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146056    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146059    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146062    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 06 19:11:10.152875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146066    2581 flags.go:64] FLAG: --runonce="false"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146070    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146073    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146076    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146079    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146082    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146085    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146088    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146092    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146095    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146098    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146100    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146104    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146107    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146111    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146114    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146119    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146122    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146125    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146129    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146132    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146135    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146138    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146141    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146144    2581 flags.go:64] FLAG: --v="2"
Sep 06 19:11:10.153503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146148    2581 flags.go:64] FLAG: --version="false"
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146152    2581 flags.go:64] FLAG: --vmodule=""
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146155    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.146159    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146307    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146311    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146314    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146317    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146321    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146324    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146326    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146329    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146332    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146334    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146337    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146339    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146342    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146344    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146347    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146351    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146353    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 19:11:10.154153 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146355    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146359    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146361    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146363    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146367    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146371    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146374    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146377    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146379    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146382    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146401    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146404    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146407    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146410    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146413    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146415    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146418    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146420    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146423    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 19:11:10.154722 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146426    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146428    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146431    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146433    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146436    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146438    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146440    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146443    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146445    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146448    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146451    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146455    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146458    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146461    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146463    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146466    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146469    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146472    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146474    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 19:11:10.155197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146478    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146481    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146484    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146487    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146489    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146492    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146494    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146497    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146499    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146502    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146504    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146506    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146509    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146511    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146514    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146516    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146518    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146521    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146523    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146526    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 19:11:10.155688 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146528    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146530    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146533    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146536    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146538    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146541    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146544    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146546    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146548    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146551    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146554    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146556    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146559    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146561    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146563    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146566    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146568    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146571    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146573    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146576    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 19:11:10.156197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146578    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146580    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146583    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.146585    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.147528    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.155433    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.155448    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.156360    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.156366    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156446    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156452    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156456    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156459    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156462    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156465    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 19:11:10.156702 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156467    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156470    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156473    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156476    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156478    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156481    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156493    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156502    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156505    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156507    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156513    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156517    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156521    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156524    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156528    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156531    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156534    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156537    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 19:11:10.157075 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156540    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156543    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156547    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156550    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156553    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156556    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156559    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156562    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156565    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156568    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156571    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156573    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156576    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156578    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156581    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156583    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156585    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156588    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156591    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156593    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 19:11:10.157540 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156595    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156604    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156606    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156609    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156611    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156614    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156617    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156619    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156622    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156624    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156626    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156629    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156631    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156633    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156636    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156638    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156641    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156644    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156647    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156650    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 19:11:10.158080 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156652    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156654    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156657    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156659    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156662    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156664    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156667    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156669    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156672    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156674    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156676    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156679    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156681    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156683    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156693    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156695    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156698    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156700    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156702    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156705    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 19:11:10.158642 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156708    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156711    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156713    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156716    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156718    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156720    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156723    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156725    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156728    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156731    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156734    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156736    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156739    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156741    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156744    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 19:11:10.159136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.156749    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156929    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156934    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156937    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156941    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156943    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156946    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156948    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156951    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156953    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156956    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156958    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156966    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156969    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156972    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156974    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156977    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156979    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156982    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156984    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 19:11:10.159524 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156987    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156989    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156992    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156995    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.156997    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157000    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157002    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157004    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157007    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157009    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157012    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157014    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157017    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157019    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157021    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157025    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157028    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157031    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157034    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157037    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 19:11:10.160026 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157039    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157042    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157044    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157047    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157049    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157057    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157059    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157061    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157064    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157066    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157069    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157071    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157074    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157076    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157079    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157081    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157083    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157086    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157089    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157091    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 19:11:10.160531 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157093    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157096    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157098    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157101    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157103    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157105    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157108    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157110    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157112    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157115    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157117    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157120    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157122    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157124    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157128    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157131    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157133    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157136    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157144    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 19:11:10.161009 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157146    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157149    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157151    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157154    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157156    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157159    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157161    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157164    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157166    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157169    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157171    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157173    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157176    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157178    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157180    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157183    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157185    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157189    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157192    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 19:11:10.161478 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157194    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 19:11:10.161959 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:10.157197    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 19:11:10.161959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.157202    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 19:11:10.161959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.157324    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 06 19:11:10.161959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.159799    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 06 19:11:10.161959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.160837    2581 server.go:1019] "Starting client certificate rotation"
Sep 06 19:11:10.161959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.160945    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 19:11:10.161959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.160980    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 19:11:10.163947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.163933    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 19:11:10.170337 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.170314    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 19:11:10.183903 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.183880    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 06 19:11:10.187462 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.187448    2581 log.go:25] "Validated CRI v1 image API"
Sep 06 19:11:10.188293 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.188273    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 06 19:11:10.190288 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.190271    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 19:11:10.192643 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.192624    2581 fs.go:135] Filesystem UUIDs: map[2b83bc9d-fb1a-4669-bd42-9980782de5ce:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 aebda78d-c9e4-4bf7-b18e-b6064aea35d4:/dev/nvme0n1p3]
Sep 06 19:11:10.192694 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.192642    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 06 19:11:10.197741 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.197630    2581 manager.go:216] Machine: {Timestamp:2026-09-06 19:11:10.196246179 +0000 UTC m=+0.385374573 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101341 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ff7b1a3ca8cd2b53f9d86716477a2 SystemUUID:ec2ff7b1-a3ca-8cd2-b53f-9d86716477a2 BootID:34187d5f-07bc-4021-9841-d39beeddb7fe Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:95:b1:30:33:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:95:b1:30:33:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:62:48:09:c3:c9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 06 19:11:10.197741 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.197738    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 06 19:11:10.197878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.197865    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 06 19:11:10.201515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.201490    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 06 19:11:10.201658 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.201517    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-200.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 06 19:11:10.202379 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.202369    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 06 19:11:10.202428 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.202381    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 06 19:11:10.202428 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.202406    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 19:11:10.202428 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.202423    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 19:11:10.203655 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.203644    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 06 19:11:10.208317 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.208304    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 19:11:10.208473 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.208463    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 06 19:11:10.208506 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.208494    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 06 19:11:10.208532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.208512    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 06 19:11:10.208532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.208523    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 06 19:11:10.208532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.208532    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 06 19:11:10.208622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.208548    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 06 19:11:10.208996 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.208984    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 19:11:10.209045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.209003    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 19:11:10.211180 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.211148    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jdrrc"
Sep 06 19:11:10.211483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.211466    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 06 19:11:10.215274 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.215244    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 06 19:11:10.215361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.215280    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 06 19:11:10.216876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216863    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 06 19:11:10.216920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216883    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 06 19:11:10.216920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216889    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 06 19:11:10.216920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216895    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 06 19:11:10.216920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216903    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 06 19:11:10.216920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216908    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 06 19:11:10.216920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216922    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 06 19:11:10.217074 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216928    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 06 19:11:10.217074 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216935    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 06 19:11:10.217074 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216942    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 06 19:11:10.217074 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216950    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 06 19:11:10.217074 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.216956    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 06 19:11:10.217942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.217923    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 06 19:11:10.217942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.217936    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 06 19:11:10.218232 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.218212    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jdrrc"
Sep 06 19:11:10.220054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.220036    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-200.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 06 19:11:10.220131 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.220055    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-200.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 06 19:11:10.220131 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.220090    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 06 19:11:10.222179 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.222164    2581 server.go:1268] "Started kubelet"
Sep 06 19:11:10.222252 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.222228    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 06 19:11:10.222301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.222241    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 06 19:11:10.222345 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.222312    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 06 19:11:10.223067 ip-10-0-135-200 systemd[1]: Started Kubernetes Kubelet.
Sep 06 19:11:10.224495 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.224481    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 06 19:11:10.224594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.224579    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 06 19:11:10.227487 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.227467    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 06 19:11:10.229999 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.229975    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 06 19:11:10.230649 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.230633    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 06 19:11:10.230776 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.230740    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:10.231408 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.231376    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 06 19:11:10.231543 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.231379    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 06 19:11:10.231622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.231548    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 06 19:11:10.231711 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.231697    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 06 19:11:10.231711 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.231712    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 06 19:11:10.232700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.232124    2581 factory.go:55] Registering systemd factory
Sep 06 19:11:10.232700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.232142    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 06 19:11:10.232700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.232595    2581 factory.go:153] Registering CRI-O factory
Sep 06 19:11:10.232700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.232608    2581 factory.go:223] Registration of the crio container factory successfully
Sep 06 19:11:10.232700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.232679    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 06 19:11:10.232940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.232731    2581 factory.go:103] Registering Raw factory
Sep 06 19:11:10.232940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.232750    2581 manager.go:1196] Started watching for new ooms in manager
Sep 06 19:11:10.232940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.232870    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 19:11:10.233344 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.233195    2581 manager.go:349] Starting recovery of all containers
Sep 06 19:11:10.237214 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.237190    2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-200.ec2.internal\" not found" node="ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.244515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.244497    2581 manager.go:354] Recovery completed
Sep 06 19:11:10.248861 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.248847    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 19:11:10.250903 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.250887    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 19:11:10.250968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.250917    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 19:11:10.250968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.250928    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientPID"
Sep 06 19:11:10.251550 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.251537    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 06 19:11:10.251590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.251550    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 06 19:11:10.251590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.251542    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 19:11:10.251590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.251566    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 19:11:10.252179 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.252167    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 19:11:10.252216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.252192    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 19:11:10.252216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.252203    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientPID"
Sep 06 19:11:10.253712 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.253700    2581 policy_none.go:49] "None policy: Start"
Sep 06 19:11:10.253756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.253716    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 06 19:11:10.253756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.253727    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 06 19:11:10.255687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.255676    2581 policy_none.go:47] "Start"
Sep 06 19:11:10.310125 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.309977    2581 manager.go:337] "Starting Device Plugin manager"
Sep 06 19:11:10.324241 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.310170    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 06 19:11:10.324241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.310188    2581 server.go:87] "Starting device plugin registration server"
Sep 06 19:11:10.324241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.310545    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 06 19:11:10.324241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.310557    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 06 19:11:10.324241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.310683    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 06 19:11:10.324241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.310764    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 06 19:11:10.324241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.310773    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 06 19:11:10.324241 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.311401    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 06 19:11:10.324241 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.311454    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:10.349039 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.348999    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 06 19:11:10.350417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.350378    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 06 19:11:10.350475 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.350458    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 06 19:11:10.350510 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.350487    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 06 19:11:10.350599 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.350585    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 06 19:11:10.352562 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.352541    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 19:11:10.410808 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.410703    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 19:11:10.411824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.411809    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 19:11:10.411885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.411841    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 19:11:10.411885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.411851    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientPID"
Sep 06 19:11:10.411885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.411874    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.419922 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.419906    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.419993 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.419930    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-200.ec2.internal\": node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:10.437576 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.437546    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:10.451652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.451619    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal"]
Sep 06 19:11:10.451753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.451737    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 19:11:10.454224 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.454201    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 19:11:10.454320 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.454235    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 19:11:10.454320 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.454245    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientPID"
Sep 06 19:11:10.456534 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.456521    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 19:11:10.456622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.456606    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.456679 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.456647    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 19:11:10.457468 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.457447    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 19:11:10.457468 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.457473    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 19:11:10.457609 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.457447    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 19:11:10.457609 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.457508    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 19:11:10.457609 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.457519    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientPID"
Sep 06 19:11:10.457609 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.457482    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientPID"
Sep 06 19:11:10.459802 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.459789    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.459856 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.459814    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 19:11:10.460731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.460713    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 19:11:10.460793 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.460750    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 19:11:10.460793 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.460765    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeHasSufficientPID"
Sep 06 19:11:10.483346 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.483317    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-200.ec2.internal\" not found" node="ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.487743 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.487725    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-200.ec2.internal\" not found" node="ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.533754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.533722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f3aa7db1aef418305307be74e295076-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal\" (UID: \"0f3aa7db1aef418305307be74e295076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.533754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.533757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f3aa7db1aef418305307be74e295076-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal\" (UID: \"0f3aa7db1aef418305307be74e295076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.533892 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.533788    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e2efa84dec2db25c9bb91631edcf5d2-config\") pod \"kube-apiserver-proxy-ip-10-0-135-200.ec2.internal\" (UID: \"1e2efa84dec2db25c9bb91631edcf5d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.537849 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.537827    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:10.634754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.634717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f3aa7db1aef418305307be74e295076-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal\" (UID: \"0f3aa7db1aef418305307be74e295076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.634754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.634753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f3aa7db1aef418305307be74e295076-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal\" (UID: \"0f3aa7db1aef418305307be74e295076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.634881 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.634768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e2efa84dec2db25c9bb91631edcf5d2-config\") pod \"kube-apiserver-proxy-ip-10-0-135-200.ec2.internal\" (UID: \"1e2efa84dec2db25c9bb91631edcf5d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.634881 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.634810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f3aa7db1aef418305307be74e295076-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal\" (UID: \"0f3aa7db1aef418305307be74e295076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.634881 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.634815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f3aa7db1aef418305307be74e295076-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal\" (UID: \"0f3aa7db1aef418305307be74e295076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.634881 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.634850    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e2efa84dec2db25c9bb91631edcf5d2-config\") pod \"kube-apiserver-proxy-ip-10-0-135-200.ec2.internal\" (UID: \"1e2efa84dec2db25c9bb91631edcf5d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.638851 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.638828    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:10.739741 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.739670    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:10.785871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.785841    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.791122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:10.790847    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:10.840759 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.840722    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:10.941249 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:10.941225    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:11.041910 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:11.041823    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:11.142459 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:11.142430    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:11.160621 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.160588    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 06 19:11:11.160791 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.160768    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 19:11:11.160845 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.160777    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 19:11:11.206942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.206915    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 19:11:11.224527 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.224483    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-05 19:06:12 +0000 UTC" deadline="2028-02-17 03:24:33.357679016 +0000 UTC"
Sep 06 19:11:11.224527 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.224520    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12680h13m22.13316159s"
Sep 06 19:11:11.242933 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:11.242900    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-200.ec2.internal\" not found"
Sep 06 19:11:11.271580 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.271555    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 19:11:11.331249 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.331141    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:11.341169 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.341144    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 19:11:11.342193 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.342177    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal"
Sep 06 19:11:11.350476 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.350455    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 19:11:11.381321 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:11.381270    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f3aa7db1aef418305307be74e295076.slice/crio-fbc286937696f242ce393a899ed6e16e5bf6c687a2304300ed8e56500b988712 WatchSource:0}: Error finding container fbc286937696f242ce393a899ed6e16e5bf6c687a2304300ed8e56500b988712: Status 404 returned error can't find the container with id fbc286937696f242ce393a899ed6e16e5bf6c687a2304300ed8e56500b988712
Sep 06 19:11:11.381650 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:11.381614    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e2efa84dec2db25c9bb91631edcf5d2.slice/crio-732fbab0495cfcbe04de22e238b861e38bda9f6b9675ea3c07024791c0cde321 WatchSource:0}: Error finding container 732fbab0495cfcbe04de22e238b861e38bda9f6b9675ea3c07024791c0cde321: Status 404 returned error can't find the container with id 732fbab0495cfcbe04de22e238b861e38bda9f6b9675ea3c07024791c0cde321
Sep 06 19:11:11.387336 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:11.387320    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 19:11:14.118331 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.118293    2581 apiserver.go:52] "Watching apiserver"
Sep 06 19:11:14.125490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.125465    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 06 19:11:14.125876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.125849    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-ntzdl","kube-system/global-pull-secret-syncer-7djzc","kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx","openshift-dns/node-resolver-sf5l2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal","openshift-multus/network-metrics-daemon-xm5z7","openshift-network-diagnostics/network-check-target-jt8wx","openshift-ovn-kubernetes/ovnkube-node-r4frj","kube-system/konnectivity-agent-97wm4","openshift-cluster-node-tuning-operator/tuned-5h2pq","openshift-image-registry/node-ca-gkckz","openshift-multus/multus-additional-cni-plugins-gb7wn","openshift-multus/multus-gw6vb"]
Sep 06 19:11:14.128602 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.128584    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:14.128697 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.128665    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:14.130849 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.130822    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.132872 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.132850    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 06 19:11:14.132979 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.132855    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 06 19:11:14.133045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.133022    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 06 19:11:14.133193 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.133178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-crhzb\""
Sep 06 19:11:14.133247 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.133234    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 06 19:11:14.133341 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.133321    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 06 19:11:14.133392 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.133368    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.135158 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.135126    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6dkxz\""
Sep 06 19:11:14.135249 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.135165    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 06 19:11:14.135388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.135370    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.137078 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.137027    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 06 19:11:14.137398 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.137378    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wpgkz\""
Sep 06 19:11:14.137505 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.137416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 06 19:11:14.137578 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.137420    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 06 19:11:14.137736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.137710    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.139944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.139396    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 06 19:11:14.139944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.139554    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 06 19:11:14.139944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.139620    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4ztjq\""
Sep 06 19:11:14.139944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.139626    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 06 19:11:14.139944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.139834    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:14.139944 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.139895    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:14.144127 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.142798    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.144874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.144853    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 06 19:11:14.145275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.145076    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 06 19:11:14.145397 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.145377    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nkn8c\""
Sep 06 19:11:14.145397 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.145392    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 06 19:11:14.145498 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.145420    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 06 19:11:14.145730 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.145714    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 06 19:11:14.145813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.145779    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 06 19:11:14.145871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.145713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.147611 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.147593    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 06 19:11:14.147827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.147810    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pgb29\""
Sep 06 19:11:14.148017 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.148004    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 06 19:11:14.148200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.148183    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 06 19:11:14.148508 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.148487    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 19:11:14.148714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.148699    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.148799 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.148768    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:14.150256 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.150239    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 19:11:14.150843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.150828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:14.152114 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da3d32c4-7c34-492c-9d9e-70f03708d30d-ovnkube-script-lib\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.152192 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152122    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tggm\" (UniqueName: \"kubernetes.io/projected/88928f30-fd24-43b0-9f56-3eb2192a0784-kube-api-access-5tggm\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.152192 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-kubelet\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.152192 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88928f30-fd24-43b0-9f56-3eb2192a0784-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152198    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-var-lib-cni-multus\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-systemd-units\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152240    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88928f30-fd24-43b0-9f56-3eb2192a0784-cni-binary-copy\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152286    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-cni-dir\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-run-multus-certs\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmhzs\" (UniqueName: \"kubernetes.io/projected/fae72b4c-8fa1-4c47-a810-601cdffc6722-kube-api-access-dmhzs\") pod \"node-resolver-sf5l2\" (UID: \"fae72b4c-8fa1-4c47-a810-601cdffc6722\") " pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-var-lib-openvswitch\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-run-ovn\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152379    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-log-socket\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152399    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-run-ovn-kubernetes\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152417    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-cni-bin\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9e352ae-eef3-4efd-9219-050e2aa51c2f-cni-binary-copy\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152564    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-cnibin\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5w4d\" (UniqueName: \"kubernetes.io/projected/f9e352ae-eef3-4efd-9219-050e2aa51c2f-kube-api-access-v5w4d\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152620    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 06 19:11:14.152871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152625    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152630    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-57bcl\""
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152628    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-run-netns\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152780    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fae72b4c-8fa1-4c47-a810-601cdffc6722-hosts-file\") pod \"node-resolver-sf5l2\" (UID: \"fae72b4c-8fa1-4c47-a810-601cdffc6722\") " pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da3d32c4-7c34-492c-9d9e-70f03708d30d-ovnkube-config\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-hostroot\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-cni-netd\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88928f30-fd24-43b0-9f56-3eb2192a0784-system-cni-dir\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-system-cni-dir\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.152971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-run-k8s-cni-cncf-io\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-daemon-config\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v7hv\" (UniqueName: \"kubernetes.io/projected/88608533-3551-4eaf-8716-5757cd3108c0-kube-api-access-4v7hv\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-device-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-slash\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153151    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-run-systemd\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.153820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153182    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da3d32c4-7c34-492c-9d9e-70f03708d30d-env-overrides\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjczd\" (UniqueName: \"kubernetes.io/projected/da3d32c4-7c34-492c-9d9e-70f03708d30d-kube-api-access-rjczd\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-registration-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-sys-fs\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-run-openvswitch\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153285    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da3d32c4-7c34-492c-9d9e-70f03708d30d-ovn-node-metrics-cert\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-socket-dir-parent\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-etc-kubernetes\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153340    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-etc-selinux\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153363    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88928f30-fd24-43b0-9f56-3eb2192a0784-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-var-lib-cni-bin\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153401    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-conf-dir\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-var-lib-kubelet\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153451    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-socket-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153464    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-run-netns\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-etc-openvswitch\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.154565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-node-log\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.155105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88928f30-fd24-43b0-9f56-3eb2192a0784-cnibin\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.155105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88928f30-fd24-43b0-9f56-3eb2192a0784-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.155105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153653    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.155105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs428\" (UniqueName: \"kubernetes.io/projected/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-kube-api-access-rs428\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.155105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.153750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae72b4c-8fa1-4c47-a810-601cdffc6722-tmp-dir\") pod \"node-resolver-sf5l2\" (UID: \"fae72b4c-8fa1-4c47-a810-601cdffc6722\") " pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.155500 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.155457    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 06 19:11:14.155944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.155569    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 06 19:11:14.155944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.155799    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8wxbg\""
Sep 06 19:11:14.156181 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.156165    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.158255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.158238    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 06 19:11:14.158357 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.158266    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 06 19:11:14.158440 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.158417    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 06 19:11:14.158598 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.158580    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x5qlk\""
Sep 06 19:11:14.168908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.168881    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8txjl"
Sep 06 19:11:14.175431 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.175409    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8txjl"
Sep 06 19:11:14.241223 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.241193    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 06 19:11:14.254918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.254885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88928f30-fd24-43b0-9f56-3eb2192a0784-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.255102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.254935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e976685-77f4-4474-8657-13d9a7e0122e-etc-tuned\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.255102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.254959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs428\" (UniqueName: \"kubernetes.io/projected/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-kube-api-access-rs428\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.255102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.254986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae72b4c-8fa1-4c47-a810-601cdffc6722-tmp-dir\") pod \"node-resolver-sf5l2\" (UID: \"fae72b4c-8fa1-4c47-a810-601cdffc6722\") " pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.255102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da3d32c4-7c34-492c-9d9e-70f03708d30d-ovnkube-script-lib\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.255400 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255376    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae72b4c-8fa1-4c47-a810-601cdffc6722-tmp-dir\") pod \"node-resolver-sf5l2\" (UID: \"fae72b4c-8fa1-4c47-a810-601cdffc6722\") " pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.255460 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tggm\" (UniqueName: \"kubernetes.io/projected/88928f30-fd24-43b0-9f56-3eb2192a0784-kube-api-access-5tggm\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.255542 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-host\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.255542 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-kubelet\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.255542 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255501    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88928f30-fd24-43b0-9f56-3eb2192a0784-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.255542 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255526    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-var-lib-cni-multus\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.255542 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f578167d-aa31-47e9-841f-75928decb2e1-konnectivity-ca\") pod \"konnectivity-agent-97wm4\" (UID: \"f578167d-aa31-47e9-841f-75928decb2e1\") " pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e976685-77f4-4474-8657-13d9a7e0122e-tmp\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b36afd6-1df8-49b6-ba26-4a7e05a5dd38-host\") pod \"node-ca-gkckz\" (UID: \"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38\") " pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255607    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88928f30-fd24-43b0-9f56-3eb2192a0784-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-systemd-units\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255693    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-kubelet\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255676    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-systemd-units\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88928f30-fd24-43b0-9f56-3eb2192a0784-cni-binary-copy\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-cni-dir\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da3d32c4-7c34-492c-9d9e-70f03708d30d-ovnkube-script-lib\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-run-multus-certs\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.255778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-run-multus-certs\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-sysconfig\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmhzs\" (UniqueName: \"kubernetes.io/projected/fae72b4c-8fa1-4c47-a810-601cdffc6722-kube-api-access-dmhzs\") pod \"node-resolver-sf5l2\" (UID: \"fae72b4c-8fa1-4c47-a810-601cdffc6722\") " pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255839    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-cni-dir\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-var-lib-openvswitch\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.255897    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-var-lib-openvswitch\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f578167d-aa31-47e9-841f-75928decb2e1-agent-certs\") pod \"konnectivity-agent-97wm4\" (UID: \"f578167d-aa31-47e9-841f-75928decb2e1\") " pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256015    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-var-lib-cni-multus\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256014    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88928f30-fd24-43b0-9f56-3eb2192a0784-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256028    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-sysctl-d\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256020    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88928f30-fd24-43b0-9f56-3eb2192a0784-cni-binary-copy\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-run-ovn\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-log-socket\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-run-ovn-kubernetes\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-cni-bin\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9e352ae-eef3-4efd-9219-050e2aa51c2f-cni-binary-copy\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-run-ovn\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3513c1a-3363-4136-a183-d4c4dafab844-iptables-alerter-script\") pod \"iptables-alerter-ntzdl\" (UID: \"d3513c1a-3363-4136-a183-d4c4dafab844\") " pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v9sq\" (UniqueName: \"kubernetes.io/projected/d3513c1a-3363-4136-a183-d4c4dafab844-kube-api-access-2v9sq\") pod \"iptables-alerter-ntzdl\" (UID: \"d3513c1a-3363-4136-a183-d4c4dafab844\") " pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256208    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-cni-bin\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-log-socket\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-kubelet-config\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-cnibin\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256268    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-run-ovn-kubernetes\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256313    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5w4d\" (UniqueName: \"kubernetes.io/projected/f9e352ae-eef3-4efd-9219-050e2aa51c2f-kube-api-access-v5w4d\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-kubernetes\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-cnibin\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256375    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-run-netns\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256403    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-systemd\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.256687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256425    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256453    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-run-netns\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fae72b4c-8fa1-4c47-a810-601cdffc6722-hosts-file\") pod \"node-resolver-sf5l2\" (UID: \"fae72b4c-8fa1-4c47-a810-601cdffc6722\") " pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da3d32c4-7c34-492c-9d9e-70f03708d30d-ovnkube-config\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-hostroot\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-sys\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fae72b4c-8fa1-4c47-a810-601cdffc6722-hosts-file\") pod \"node-resolver-sf5l2\" (UID: \"fae72b4c-8fa1-4c47-a810-601cdffc6722\") " pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-hostroot\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-cni-netd\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-cni-netd\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.256653    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256644    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9e352ae-eef3-4efd-9219-050e2aa51c2f-cni-binary-copy\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256665    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88928f30-fd24-43b0-9f56-3eb2192a0784-system-cni-dir\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256698    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88928f30-fd24-43b0-9f56-3eb2192a0784-system-cni-dir\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.256748    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs podName:88608533-3551-4eaf-8716-5757cd3108c0 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:14.75672279 +0000 UTC m=+3.037175585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs") pod "network-metrics-daemon-xm5z7" (UID: "88608533-3551-4eaf-8716-5757cd3108c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzbs5\" (UniqueName: \"kubernetes.io/projected/9b36afd6-1df8-49b6-ba26-4a7e05a5dd38-kube-api-access-zzbs5\") pod \"node-ca-gkckz\" (UID: \"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38\") " pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.257486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-system-cni-dir\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-run-k8s-cni-cncf-io\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256841    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-daemon-config\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-var-lib-kubelet\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-run-k8s-cni-cncf-io\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v7hv\" (UniqueName: \"kubernetes.io/projected/88608533-3551-4eaf-8716-5757cd3108c0-kube-api-access-4v7hv\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256951    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-dbus\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-system-cni-dir\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.256995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-device-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da3d32c4-7c34-492c-9d9e-70f03708d30d-ovnkube-config\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-slash\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-run-systemd\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-device-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-run-systemd\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257101    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da3d32c4-7c34-492c-9d9e-70f03708d30d-env-overrides\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-slash\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjczd\" (UniqueName: \"kubernetes.io/projected/da3d32c4-7c34-492c-9d9e-70f03708d30d-kube-api-access-rjczd\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.258251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257187    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3513c1a-3363-4136-a183-d4c4dafab844-host-slash\") pod \"iptables-alerter-ntzdl\" (UID: \"d3513c1a-3363-4136-a183-d4c4dafab844\") " pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-registration-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-sys-fs\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-run-openvswitch\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da3d32c4-7c34-492c-9d9e-70f03708d30d-ovn-node-metrics-cert\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-daemon-config\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-socket-dir-parent\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-registration-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257353    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-etc-kubernetes\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-sys-fs\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-run-openvswitch\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-sysctl-conf\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257396    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-socket-dir-parent\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-run\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-etc-kubernetes\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-etc-selinux\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.259011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88928f30-fd24-43b0-9f56-3eb2192a0784-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da3d32c4-7c34-492c-9d9e-70f03708d30d-env-overrides\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257536    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-etc-selinux\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-var-lib-cni-bin\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257556    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-var-lib-cni-bin\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257581    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-conf-dir\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-modprobe-d\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-multus-conf-dir\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88928f30-fd24-43b0-9f56-3eb2192a0784-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-lib-modules\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257706    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k56qn\" (UniqueName: \"kubernetes.io/projected/6e976685-77f4-4474-8657-13d9a7e0122e-kube-api-access-k56qn\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257718    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257735    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-var-lib-kubelet\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257776    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9e352ae-eef3-4efd-9219-050e2aa51c2f-host-var-lib-kubelet\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9b36afd6-1df8-49b6-ba26-4a7e05a5dd38-serviceca\") pod \"node-ca-gkckz\" (UID: \"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38\") " pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-socket-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-run-netns\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-etc-openvswitch\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.259736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-node-log\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.260352 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88928f30-fd24-43b0-9f56-3eb2192a0784-cnibin\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.260352 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-etc-openvswitch\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.260352 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-host-run-netns\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.260352 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-socket-dir\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.260352 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257955    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da3d32c4-7c34-492c-9d9e-70f03708d30d-node-log\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.260352 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.257981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88928f30-fd24-43b0-9f56-3eb2192a0784-cnibin\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.261965 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.261939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da3d32c4-7c34-492c-9d9e-70f03708d30d-ovn-node-metrics-cert\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.262656 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.262627    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 19:11:14.262656 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.262651    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 19:11:14.262797 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.262664    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kdswv for pod openshift-network-diagnostics/network-check-target-jt8wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:14.262797 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.262715    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv podName:bfc0b5e6-446a-482f-8ea3-087c734a417a nodeName:}" failed. No retries permitted until 2026-09-06 19:11:14.762698473 +0000 UTC m=+3.043151246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kdswv" (UniqueName: "kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv") pod "network-check-target-jt8wx" (UID: "bfc0b5e6-446a-482f-8ea3-087c734a417a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:14.263905 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.263846    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tggm\" (UniqueName: \"kubernetes.io/projected/88928f30-fd24-43b0-9f56-3eb2192a0784-kube-api-access-5tggm\") pod \"multus-additional-cni-plugins-gb7wn\" (UID: \"88928f30-fd24-43b0-9f56-3eb2192a0784\") " pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.264017 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.263962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs428\" (UniqueName: \"kubernetes.io/projected/2a3f97c5-5025-4cd1-84e4-bc2f9f406918-kube-api-access-rs428\") pod \"aws-ebs-csi-driver-node-hb6hx\" (UID: \"2a3f97c5-5025-4cd1-84e4-bc2f9f406918\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.264253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.264223    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmhzs\" (UniqueName: \"kubernetes.io/projected/fae72b4c-8fa1-4c47-a810-601cdffc6722-kube-api-access-dmhzs\") pod \"node-resolver-sf5l2\" (UID: \"fae72b4c-8fa1-4c47-a810-601cdffc6722\") " pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.265879 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.265750    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v7hv\" (UniqueName: \"kubernetes.io/projected/88608533-3551-4eaf-8716-5757cd3108c0-kube-api-access-4v7hv\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:14.266359 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.266339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5w4d\" (UniqueName: \"kubernetes.io/projected/f9e352ae-eef3-4efd-9219-050e2aa51c2f-kube-api-access-v5w4d\") pod \"multus-gw6vb\" (UID: \"f9e352ae-eef3-4efd-9219-050e2aa51c2f\") " pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.266775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.266729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal" event={"ID":"0f3aa7db1aef418305307be74e295076","Type":"ContainerStarted","Data":"fbc286937696f242ce393a899ed6e16e5bf6c687a2304300ed8e56500b988712"}
Sep 06 19:11:14.267403 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.267382    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjczd\" (UniqueName: \"kubernetes.io/projected/da3d32c4-7c34-492c-9d9e-70f03708d30d-kube-api-access-rjczd\") pod \"ovnkube-node-r4frj\" (UID: \"da3d32c4-7c34-492c-9d9e-70f03708d30d\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.268567 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.268492    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal" event={"ID":"1e2efa84dec2db25c9bb91631edcf5d2","Type":"ContainerStarted","Data":"732fbab0495cfcbe04de22e238b861e38bda9f6b9675ea3c07024791c0cde321"}
Sep 06 19:11:14.358774 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-sysctl-conf\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.358774 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-run\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-modprobe-d\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-lib-modules\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358868    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k56qn\" (UniqueName: \"kubernetes.io/projected/6e976685-77f4-4474-8657-13d9a7e0122e-kube-api-access-k56qn\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9b36afd6-1df8-49b6-ba26-4a7e05a5dd38-serviceca\") pod \"node-ca-gkckz\" (UID: \"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38\") " pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358896    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-run\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e976685-77f4-4474-8657-13d9a7e0122e-etc-tuned\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358943    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-sysctl-conf\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-host\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.358994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f578167d-aa31-47e9-841f-75928decb2e1-konnectivity-ca\") pod \"konnectivity-agent-97wm4\" (UID: \"f578167d-aa31-47e9-841f-75928decb2e1\") " pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e976685-77f4-4474-8657-13d9a7e0122e-tmp\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-modprobe-d\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b36afd6-1df8-49b6-ba26-4a7e05a5dd38-host\") pod \"node-ca-gkckz\" (UID: \"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38\") " pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b36afd6-1df8-49b6-ba26-4a7e05a5dd38-host\") pod \"node-ca-gkckz\" (UID: \"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38\") " pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-sysconfig\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359110    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f578167d-aa31-47e9-841f-75928decb2e1-agent-certs\") pod \"konnectivity-agent-97wm4\" (UID: \"f578167d-aa31-47e9-841f-75928decb2e1\") " pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-sysctl-d\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3513c1a-3363-4136-a183-d4c4dafab844-iptables-alerter-script\") pod \"iptables-alerter-ntzdl\" (UID: \"d3513c1a-3363-4136-a183-d4c4dafab844\") " pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v9sq\" (UniqueName: \"kubernetes.io/projected/d3513c1a-3363-4136-a183-d4c4dafab844-kube-api-access-2v9sq\") pod \"iptables-alerter-ntzdl\" (UID: \"d3513c1a-3363-4136-a183-d4c4dafab844\") " pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-kubelet-config\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-host\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-kubernetes\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-systemd\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-kubernetes\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-lib-modules\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-sys\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-sysconfig\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzbs5\" (UniqueName: \"kubernetes.io/projected/9b36afd6-1df8-49b6-ba26-4a7e05a5dd38-kube-api-access-zzbs5\") pod \"node-ca-gkckz\" (UID: \"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38\") " pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-var-lib-kubelet\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359501    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-dbus\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.359610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3513c1a-3363-4136-a183-d4c4dafab844-host-slash\") pod \"iptables-alerter-ntzdl\" (UID: \"d3513c1a-3363-4136-a183-d4c4dafab844\") " pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359720    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f578167d-aa31-47e9-841f-75928decb2e1-konnectivity-ca\") pod \"konnectivity-agent-97wm4\" (UID: \"f578167d-aa31-47e9-841f-75928decb2e1\") " pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.359770    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-kubelet-config\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.359901    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret podName:5d5de6d9-7cef-4edf-8164-dbb60b3089e9 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:14.859884023 +0000 UTC m=+3.140336796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret") pod "global-pull-secret-syncer-7djzc" (UID: "5d5de6d9-7cef-4edf-8164-dbb60b3089e9") : object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-sysctl-d\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.360005    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-sys\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.359503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9b36afd6-1df8-49b6-ba26-4a7e05a5dd38-serviceca\") pod \"node-ca-gkckz\" (UID: \"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38\") " pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.360022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-etc-systemd\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.360048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e976685-77f4-4474-8657-13d9a7e0122e-var-lib-kubelet\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.360087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3513c1a-3363-4136-a183-d4c4dafab844-host-slash\") pod \"iptables-alerter-ntzdl\" (UID: \"d3513c1a-3363-4136-a183-d4c4dafab844\") " pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.360099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-dbus\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.360335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.360098    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3513c1a-3363-4136-a183-d4c4dafab844-iptables-alerter-script\") pod \"iptables-alerter-ntzdl\" (UID: \"d3513c1a-3363-4136-a183-d4c4dafab844\") " pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.362501 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.362480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e976685-77f4-4474-8657-13d9a7e0122e-etc-tuned\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.362605 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.362557    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e976685-77f4-4474-8657-13d9a7e0122e-tmp\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.362697 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.362682    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f578167d-aa31-47e9-841f-75928decb2e1-agent-certs\") pod \"konnectivity-agent-97wm4\" (UID: \"f578167d-aa31-47e9-841f-75928decb2e1\") " pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:14.366832 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.366803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k56qn\" (UniqueName: \"kubernetes.io/projected/6e976685-77f4-4474-8657-13d9a7e0122e-kube-api-access-k56qn\") pod \"tuned-5h2pq\" (UID: \"6e976685-77f4-4474-8657-13d9a7e0122e\") " pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.367756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.367734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzbs5\" (UniqueName: \"kubernetes.io/projected/9b36afd6-1df8-49b6-ba26-4a7e05a5dd38-kube-api-access-zzbs5\") pod \"node-ca-gkckz\" (UID: \"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38\") " pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.367861 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.367823    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v9sq\" (UniqueName: \"kubernetes.io/projected/d3513c1a-3363-4136-a183-d4c4dafab844-kube-api-access-2v9sq\") pod \"iptables-alerter-ntzdl\" (UID: \"d3513c1a-3363-4136-a183-d4c4dafab844\") " pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.444346 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.444146    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gb7wn"
Sep 06 19:11:14.452170 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.452146    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gw6vb"
Sep 06 19:11:14.454122 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:14.454098    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88928f30_fd24_43b0_9f56_3eb2192a0784.slice/crio-f7572bb9e7417fb5b877ceb1ba8302e2836d6552f0958f1fed38738f8de67f64 WatchSource:0}: Error finding container f7572bb9e7417fb5b877ceb1ba8302e2836d6552f0958f1fed38738f8de67f64: Status 404 returned error can't find the container with id f7572bb9e7417fb5b877ceb1ba8302e2836d6552f0958f1fed38738f8de67f64
Sep 06 19:11:14.461319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.461297    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx"
Sep 06 19:11:14.461936 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:14.461917    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e352ae_eef3_4efd_9219_050e2aa51c2f.slice/crio-be93bcddf6d4de9a1e894a794a6c94ed731ff2b0aba1b76f7616e58dba093729 WatchSource:0}: Error finding container be93bcddf6d4de9a1e894a794a6c94ed731ff2b0aba1b76f7616e58dba093729: Status 404 returned error can't find the container with id be93bcddf6d4de9a1e894a794a6c94ed731ff2b0aba1b76f7616e58dba093729
Sep 06 19:11:14.466050 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.466032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sf5l2"
Sep 06 19:11:14.470575 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:14.470550    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3f97c5_5025_4cd1_84e4_bc2f9f406918.slice/crio-016d640764cc0e89103fe1a062b3bac5d02865df38f8343992b9b68eba824bc7 WatchSource:0}: Error finding container 016d640764cc0e89103fe1a062b3bac5d02865df38f8343992b9b68eba824bc7: Status 404 returned error can't find the container with id 016d640764cc0e89103fe1a062b3bac5d02865df38f8343992b9b68eba824bc7
Sep 06 19:11:14.472273 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.472178    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:14.479902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.479884    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ntzdl"
Sep 06 19:11:14.480438 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:14.480420    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3d32c4_7c34_492c_9d9e_70f03708d30d.slice/crio-24e949b744a80489545bafc1cb14ddc1ad1e7b6526e67190f05c2a95128df219 WatchSource:0}: Error finding container 24e949b744a80489545bafc1cb14ddc1ad1e7b6526e67190f05c2a95128df219: Status 404 returned error can't find the container with id 24e949b744a80489545bafc1cb14ddc1ad1e7b6526e67190f05c2a95128df219
Sep 06 19:11:14.486597 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.486579    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:14.489311 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:14.489237    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3513c1a_3363_4136_a183_d4c4dafab844.slice/crio-fc114d7029848a3169774d072ba01d0c4573a5bd98b08257d18f969bb8250c8f WatchSource:0}: Error finding container fc114d7029848a3169774d072ba01d0c4573a5bd98b08257d18f969bb8250c8f: Status 404 returned error can't find the container with id fc114d7029848a3169774d072ba01d0c4573a5bd98b08257d18f969bb8250c8f
Sep 06 19:11:14.491660 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.491631    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 19:11:14.492666 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.492648    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5h2pq"
Sep 06 19:11:14.497595 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.497473    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gkckz"
Sep 06 19:11:14.504773 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:14.504625    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e976685_77f4_4474_8657_13d9a7e0122e.slice/crio-bbd0d2768c424e4da58794bf3ceab0c0aa22dfe3963f688cf4883ff3c2c2ba68 WatchSource:0}: Error finding container bbd0d2768c424e4da58794bf3ceab0c0aa22dfe3963f688cf4883ff3c2c2ba68: Status 404 returned error can't find the container with id bbd0d2768c424e4da58794bf3ceab0c0aa22dfe3963f688cf4883ff3c2c2ba68
Sep 06 19:11:14.507782 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:14.507733    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b36afd6_1df8_49b6_ba26_4a7e05a5dd38.slice/crio-11bffdee9c154e93fb9f3db7c3a4fb9df671446dd1bb0aeb3da8129557256b83 WatchSource:0}: Error finding container 11bffdee9c154e93fb9f3db7c3a4fb9df671446dd1bb0aeb3da8129557256b83: Status 404 returned error can't find the container with id 11bffdee9c154e93fb9f3db7c3a4fb9df671446dd1bb0aeb3da8129557256b83
Sep 06 19:11:14.762865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.762834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:14.763022 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.762883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:14.763022 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.763004    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 19:11:14.763125 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.763025    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 19:11:14.763125 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.763036    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kdswv for pod openshift-network-diagnostics/network-check-target-jt8wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:14.763125 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.763048    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:14.763125 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.763099    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv podName:bfc0b5e6-446a-482f-8ea3-087c734a417a nodeName:}" failed. No retries permitted until 2026-09-06 19:11:15.763083845 +0000 UTC m=+4.043536621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kdswv" (UniqueName: "kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv") pod "network-check-target-jt8wx" (UID: "bfc0b5e6-446a-482f-8ea3-087c734a417a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:14.763125 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.763114    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs podName:88608533-3551-4eaf-8716-5757cd3108c0 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:15.763107786 +0000 UTC m=+4.043560558 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs") pod "network-metrics-daemon-xm5z7" (UID: "88608533-3551-4eaf-8716-5757cd3108c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:14.863370 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:14.863339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:14.863550 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.863511    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:14.863612 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:14.863580    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret podName:5d5de6d9-7cef-4edf-8164-dbb60b3089e9 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:15.863560804 +0000 UTC m=+4.144013593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret") pod "global-pull-secret-syncer-7djzc" (UID: "5d5de6d9-7cef-4edf-8164-dbb60b3089e9") : object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:15.179107 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.178918    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 19:06:14 +0000 UTC" deadline="2028-02-09 00:35:52.107039746 +0000 UTC"
Sep 06 19:11:15.179107 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.178982    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12485h24m36.928062068s"
Sep 06 19:11:15.260287 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.260224    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:15.260473 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:15.260364    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:15.286083 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.286024    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerStarted","Data":"24e949b744a80489545bafc1cb14ddc1ad1e7b6526e67190f05c2a95128df219"}
Sep 06 19:11:15.302915 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.302875    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx" event={"ID":"2a3f97c5-5025-4cd1-84e4-bc2f9f406918","Type":"ContainerStarted","Data":"016d640764cc0e89103fe1a062b3bac5d02865df38f8343992b9b68eba824bc7"}
Sep 06 19:11:15.316751 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.316703    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gw6vb" event={"ID":"f9e352ae-eef3-4efd-9219-050e2aa51c2f","Type":"ContainerStarted","Data":"be93bcddf6d4de9a1e894a794a6c94ed731ff2b0aba1b76f7616e58dba093729"}
Sep 06 19:11:15.328084 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.326118    2581 generic.go:356] "Generic (PLEG): container finished" podID="0f3aa7db1aef418305307be74e295076" containerID="78dd2cc4d585fbdb64f775b763e00e027a46f912c9bb438855ddcff3fd7dfcf3" exitCode=0
Sep 06 19:11:15.328084 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.326158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal" event={"ID":"0f3aa7db1aef418305307be74e295076","Type":"ContainerDied","Data":"78dd2cc4d585fbdb64f775b763e00e027a46f912c9bb438855ddcff3fd7dfcf3"}
Sep 06 19:11:15.347939 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.347892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gkckz" event={"ID":"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38","Type":"ContainerStarted","Data":"11bffdee9c154e93fb9f3db7c3a4fb9df671446dd1bb0aeb3da8129557256b83"}
Sep 06 19:11:15.361413 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.361365    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-97wm4" event={"ID":"f578167d-aa31-47e9-841f-75928decb2e1","Type":"ContainerStarted","Data":"5c7ac3617814cf5c5e418a4e16e3f6e8a23e01678b5bf42f3593dfa8a131f49b"}
Sep 06 19:11:15.378682 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.378603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sf5l2" event={"ID":"fae72b4c-8fa1-4c47-a810-601cdffc6722","Type":"ContainerStarted","Data":"8790135e1d55281236632cafe64caa21ef25c12638288cb7f0843c42bec6404c"}
Sep 06 19:11:15.385931 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.385822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb7wn" event={"ID":"88928f30-fd24-43b0-9f56-3eb2192a0784","Type":"ContainerStarted","Data":"f7572bb9e7417fb5b877ceb1ba8302e2836d6552f0958f1fed38738f8de67f64"}
Sep 06 19:11:15.392603 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.392510    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5h2pq" event={"ID":"6e976685-77f4-4474-8657-13d9a7e0122e","Type":"ContainerStarted","Data":"bbd0d2768c424e4da58794bf3ceab0c0aa22dfe3963f688cf4883ff3c2c2ba68"}
Sep 06 19:11:15.411652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.406658    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ntzdl" event={"ID":"d3513c1a-3363-4136-a183-d4c4dafab844","Type":"ContainerStarted","Data":"fc114d7029848a3169774d072ba01d0c4573a5bd98b08257d18f969bb8250c8f"}
Sep 06 19:11:15.773455 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.773411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:15.773647 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.773468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:15.773709 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:15.773651    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:15.773757 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:15.773721    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs podName:88608533-3551-4eaf-8716-5757cd3108c0 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:17.773701525 +0000 UTC m=+6.054154303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs") pod "network-metrics-daemon-xm5z7" (UID: "88608533-3551-4eaf-8716-5757cd3108c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:15.774198 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:15.774176    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 19:11:15.774298 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:15.774206    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 19:11:15.774298 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:15.774221    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kdswv for pod openshift-network-diagnostics/network-check-target-jt8wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:15.774298 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:15.774268    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv podName:bfc0b5e6-446a-482f-8ea3-087c734a417a nodeName:}" failed. No retries permitted until 2026-09-06 19:11:17.774252949 +0000 UTC m=+6.054705724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kdswv" (UniqueName: "kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv") pod "network-check-target-jt8wx" (UID: "bfc0b5e6-446a-482f-8ea3-087c734a417a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:15.874737 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:15.874085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:15.874737 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:15.874248    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:15.874737 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:15.874312    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret podName:5d5de6d9-7cef-4edf-8164-dbb60b3089e9 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:17.874292707 +0000 UTC m=+6.154745482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret") pod "global-pull-secret-syncer-7djzc" (UID: "5d5de6d9-7cef-4edf-8164-dbb60b3089e9") : object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:16.179313 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:16.179209    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 19:06:14 +0000 UTC" deadline="2028-05-12 05:21:07.923972574 +0000 UTC"
Sep 06 19:11:16.179313 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:16.179243    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14722h9m51.744732993s"
Sep 06 19:11:16.262022 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:16.261987    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:16.262204 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:16.262150    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:16.262700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:16.262672    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:16.262808 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:16.262785    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:16.417674 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:16.417635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal" event={"ID":"0f3aa7db1aef418305307be74e295076","Type":"ContainerStarted","Data":"830f12067154fa14e603d8b51fc791c4574d785dbddf84ddc54885a6dfbf3606"}
Sep 06 19:11:17.259713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:17.259677    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:17.260375 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:17.259815    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:17.793111 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:17.793005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:17.793111 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:17.793081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:17.793351 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:17.793238    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:17.793351 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:17.793297    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs podName:88608533-3551-4eaf-8716-5757cd3108c0 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:21.793279653 +0000 UTC m=+10.073732426 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs") pod "network-metrics-daemon-xm5z7" (UID: "88608533-3551-4eaf-8716-5757cd3108c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:17.793472 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:17.793385    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 19:11:17.793472 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:17.793400    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 19:11:17.793472 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:17.793413    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kdswv for pod openshift-network-diagnostics/network-check-target-jt8wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:17.793472 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:17.793450    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv podName:bfc0b5e6-446a-482f-8ea3-087c734a417a nodeName:}" failed. No retries permitted until 2026-09-06 19:11:21.793437532 +0000 UTC m=+10.073890324 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kdswv" (UniqueName: "kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv") pod "network-check-target-jt8wx" (UID: "bfc0b5e6-446a-482f-8ea3-087c734a417a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:17.894534 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:17.893924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:17.894534 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:17.894078    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:17.894534 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:17.894137    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret podName:5d5de6d9-7cef-4edf-8164-dbb60b3089e9 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:21.894118666 +0000 UTC m=+10.174571453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret") pod "global-pull-secret-syncer-7djzc" (UID: "5d5de6d9-7cef-4edf-8164-dbb60b3089e9") : object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:18.261373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:18.260579    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:18.261373 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:18.260737    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:18.261373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:18.261179    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:18.261373 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:18.261272    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:19.259647 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:19.259608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:19.259837 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:19.259761    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:20.260034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:20.259388    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:20.260034 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:20.259540    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:20.260034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:20.259387    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:20.260034 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:20.259974    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:21.259773 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:21.259737    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:21.259958 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:21.259875    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:21.830824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:21.830784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:21.830824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:21.830840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:21.831403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:21.830952    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:21.831403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:21.830972    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 19:11:21.831403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:21.830997    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 19:11:21.831403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:21.831011    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kdswv for pod openshift-network-diagnostics/network-check-target-jt8wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:21.831403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:21.831025    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs podName:88608533-3551-4eaf-8716-5757cd3108c0 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:29.831006253 +0000 UTC m=+18.111459029 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs") pod "network-metrics-daemon-xm5z7" (UID: "88608533-3551-4eaf-8716-5757cd3108c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:21.831403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:21.831088    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv podName:bfc0b5e6-446a-482f-8ea3-087c734a417a nodeName:}" failed. No retries permitted until 2026-09-06 19:11:29.831055859 +0000 UTC m=+18.111508648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kdswv" (UniqueName: "kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv") pod "network-check-target-jt8wx" (UID: "bfc0b5e6-446a-482f-8ea3-087c734a417a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:21.932258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:21.931623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:21.932258 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:21.931845    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:21.932258 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:21.931910    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret podName:5d5de6d9-7cef-4edf-8164-dbb60b3089e9 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:29.931891128 +0000 UTC m=+18.212343922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret") pod "global-pull-secret-syncer-7djzc" (UID: "5d5de6d9-7cef-4edf-8164-dbb60b3089e9") : object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:22.265082 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:22.264984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:22.265237 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:22.265138    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:22.265601 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:22.265450    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:22.265601 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:22.265538    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:23.259722 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:23.259688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:23.260212 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:23.259825    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:24.263950 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:24.263867    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:24.264397 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:24.263875    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:24.264397 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:24.264033    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:24.264397 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:24.264127    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:25.260384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:25.260345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:25.260585 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:25.260482    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:26.259526 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:26.259492    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:26.260100 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:26.259618    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:26.260100 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:26.259629    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:26.260100 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:26.259746    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:27.260229 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:27.260194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:27.260656 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:27.260324    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:28.260282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:28.260244    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:28.260710 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:28.260396    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:28.260710 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:28.260463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:28.260710 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:28.260577    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:29.259696 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:29.259665    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:29.259880 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:29.259781    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:29.906385 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:29.906296    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:29.906385 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:29.906372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:29.906878 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:29.906482    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:29.906878 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:29.906493    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 19:11:29.906878 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:29.906586    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 19:11:29.906878 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:29.906601    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kdswv for pod openshift-network-diagnostics/network-check-target-jt8wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:29.906878 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:29.906571    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs podName:88608533-3551-4eaf-8716-5757cd3108c0 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:45.906548743 +0000 UTC m=+34.187001518 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs") pod "network-metrics-daemon-xm5z7" (UID: "88608533-3551-4eaf-8716-5757cd3108c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:29.906878 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:29.906661    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv podName:bfc0b5e6-446a-482f-8ea3-087c734a417a nodeName:}" failed. No retries permitted until 2026-09-06 19:11:45.906644635 +0000 UTC m=+34.187097407 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kdswv" (UniqueName: "kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv") pod "network-check-target-jt8wx" (UID: "bfc0b5e6-446a-482f-8ea3-087c734a417a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:30.007359 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:30.007316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:30.007555 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:30.007470    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:30.007555 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:30.007544    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret podName:5d5de6d9-7cef-4edf-8164-dbb60b3089e9 nodeName:}" failed. No retries permitted until 2026-09-06 19:11:46.007529115 +0000 UTC m=+34.287981887 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret") pod "global-pull-secret-syncer-7djzc" (UID: "5d5de6d9-7cef-4edf-8164-dbb60b3089e9") : object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:30.260278 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:30.260157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:30.260426 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:30.260309    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:30.260426 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:30.260374    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:30.260534 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:30.260474    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:31.260209 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:31.260166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:31.260647 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:31.260320    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:32.260650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:32.260607    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:32.261102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:32.260657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:32.261102 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:32.260724    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:32.261102 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:32.260837    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:33.260020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:33.259982    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:33.260194 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:33.260120    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:34.260903 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.260473    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:34.261507 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:34.261028    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:34.261577 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.260473    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:34.261693 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:34.261666    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:34.456982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.456945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal" event={"ID":"1e2efa84dec2db25c9bb91631edcf5d2","Type":"ContainerStarted","Data":"64247679ab33ae3f1890e5e92ab6758bc8c8c64d9d90987e558c5a6953d39b20"}
Sep 06 19:11:34.458638 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.458608    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gkckz" event={"ID":"9b36afd6-1df8-49b6-ba26-4a7e05a5dd38","Type":"ContainerStarted","Data":"e65d368fed5574b1f1bf30d280a5dd3561d89315966b198d780c132f008855d7"}
Sep 06 19:11:34.460164 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.460141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-97wm4" event={"ID":"f578167d-aa31-47e9-841f-75928decb2e1","Type":"ContainerStarted","Data":"5b323aeb080b03f977a7ba3bc2cb6efb60b269378b2b9db18be49128d4e813a6"}
Sep 06 19:11:34.461703 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.461653    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sf5l2" event={"ID":"fae72b4c-8fa1-4c47-a810-601cdffc6722","Type":"ContainerStarted","Data":"b571bff1286a00c79bd38dd9d354abe265286576054640dc133d5796ba8b8590"}
Sep 06 19:11:34.463455 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.463429    2581 generic.go:356] "Generic (PLEG): container finished" podID="88928f30-fd24-43b0-9f56-3eb2192a0784" containerID="56d0b5b16014e27e4a36b0c05d856fb4af85dac6cb049ecc3458a4992ffabfd3" exitCode=0
Sep 06 19:11:34.463568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.463505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb7wn" event={"ID":"88928f30-fd24-43b0-9f56-3eb2192a0784","Type":"ContainerDied","Data":"56d0b5b16014e27e4a36b0c05d856fb4af85dac6cb049ecc3458a4992ffabfd3"}
Sep 06 19:11:34.465570 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.465550    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5h2pq" event={"ID":"6e976685-77f4-4474-8657-13d9a7e0122e","Type":"ContainerStarted","Data":"75cd52aea152620df7ac7e1693ceb6dea3c5761c9dd5da96a0334e5c0fe865c5"}
Sep 06 19:11:34.468817 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.468797    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:11:34.469227 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.469200    2581 generic.go:356] "Generic (PLEG): container finished" podID="da3d32c4-7c34-492c-9d9e-70f03708d30d" containerID="9afef782b28f252b82b189fef02cc37279b45ece9b85d9c42f2b2a0329a504e0" exitCode=1
Sep 06 19:11:34.469308 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.469268    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerStarted","Data":"f97557172787d566bcefc4f448fad6959226e3b425be82bd66b2648c9a28f00e"}
Sep 06 19:11:34.469373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.469336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerStarted","Data":"dfefe2c49560a44744cd8c96a1480062ef3fc1b4943c6f152bbc37191cf71665"}
Sep 06 19:11:34.469428 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.469388    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerStarted","Data":"5336418188db4bcbdb701e999714aa672acf8d766f23e00c298826da25e8899b"}
Sep 06 19:11:34.469491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.469423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerStarted","Data":"4fdb3a0013d8ec86834de2909911c67efa3aebb78ce7ef0171f78bffb6733d11"}
Sep 06 19:11:34.469491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.469462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerDied","Data":"9afef782b28f252b82b189fef02cc37279b45ece9b85d9c42f2b2a0329a504e0"}
Sep 06 19:11:34.469571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.469542    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerStarted","Data":"3b63822892b27e952264dcc5091f4266bcdb953a5f9a0eccdf1a99baf29469dc"}
Sep 06 19:11:34.470896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.470870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx" event={"ID":"2a3f97c5-5025-4cd1-84e4-bc2f9f406918","Type":"ContainerStarted","Data":"b400d5fe7689831801b362be40d67a5cf6e98ff613762ac0bb78a5773a7415f0"}
Sep 06 19:11:34.472445 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.472423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gw6vb" event={"ID":"f9e352ae-eef3-4efd-9219-050e2aa51c2f","Type":"ContainerStarted","Data":"2e331e2c8507e5dbd88fa7c48f50c0f79b7ef12e618a61a8f39b11905bdbc100"}
Sep 06 19:11:34.472789 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.472726    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-200.ec2.internal" podStartSLOduration=21.472716573 podStartE2EDuration="21.472716573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:34.472487537 +0000 UTC m=+22.752940330" watchObservedRunningTime="2026-09-06 19:11:34.472716573 +0000 UTC m=+22.753169368"
Sep 06 19:11:34.472911 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.472893    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-200.ec2.internal" podStartSLOduration=21.472887181 podStartE2EDuration="21.472887181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:16.433949062 +0000 UTC m=+4.714401855" watchObservedRunningTime="2026-09-06 19:11:34.472887181 +0000 UTC m=+22.753339974"
Sep 06 19:11:34.487259 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.487205    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5h2pq" podStartSLOduration=3.5060164609999998 podStartE2EDuration="22.487186029s" totalImagesPullingTime="18.981169568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:34.486569943 +0000 UTC m=+22.767022737" watchObservedRunningTime="2026-09-06 19:11:34.487186029 +0000 UTC m=+22.767638860"
Sep 06 19:11:34.502920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.502865    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gkckz" podStartSLOduration=3.526247566 podStartE2EDuration="22.502846326s" totalImagesPullingTime="18.97659876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:34.502462285 +0000 UTC m=+22.782915079" watchObservedRunningTime="2026-09-06 19:11:34.502846326 +0000 UTC m=+22.783299121"
Sep 06 19:11:34.536029 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.535983    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sf5l2" podStartSLOduration=3.527576149 podStartE2EDuration="22.535964288s" totalImagesPullingTime="19.008388139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:34.535851641 +0000 UTC m=+22.816304433" watchObservedRunningTime="2026-09-06 19:11:34.535964288 +0000 UTC m=+22.816417088"
Sep 06 19:11:34.549762 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.549704    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-97wm4" podStartSLOduration=7.378535548 podStartE2EDuration="22.549684612s" totalImagesPullingTime="15.171149064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:34.549526149 +0000 UTC m=+22.829978942" watchObservedRunningTime="2026-09-06 19:11:34.549684612 +0000 UTC m=+22.830137407"
Sep 06 19:11:34.567921 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.567866    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gw6vb" podStartSLOduration=2.954321314 podStartE2EDuration="22.567845675s" totalImagesPullingTime="19.613524361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:34.567087458 +0000 UTC m=+22.847540243" watchObservedRunningTime="2026-09-06 19:11:34.567845675 +0000 UTC m=+22.848298470"
Sep 06 19:11:34.580128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.580093    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:34.691424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.691345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:34.917446 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:34.917410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:35.114032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.113995    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 06 19:11:35.233293 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.233052    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-06T19:11:35.114008968Z","UUID":"f5b990f8-40c2-4e1d-ad44-253cd10fe1a1","Handler":null,"Name":"","Endpoint":""}
Sep 06 19:11:35.234830 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.234810    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 06 19:11:35.234957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.234841    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 06 19:11:35.260280 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.260250    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:35.260471 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:35.260370    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:35.476549 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.476517    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ntzdl" event={"ID":"d3513c1a-3363-4136-a183-d4c4dafab844","Type":"ContainerStarted","Data":"f4666df1e24f2c58eb91785c09da58be2531b74f9e4230fcb019f2bd6f252a85"}
Sep 06 19:11:35.479121 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.478742    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx" event={"ID":"2a3f97c5-5025-4cd1-84e4-bc2f9f406918","Type":"ContainerStarted","Data":"ccf4c52c22050b821c4646ce2af16ad073e63ec60f19a6c6eaea922001f7cac8"}
Sep 06 19:11:35.489861 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.489769    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ntzdl" podStartSLOduration=4.526313822 podStartE2EDuration="23.489748006s" totalImagesPullingTime="18.963434184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:35.489047888 +0000 UTC m=+23.769500703" watchObservedRunningTime="2026-09-06 19:11:35.489748006 +0000 UTC m=+23.770200802"
Sep 06 19:11:35.607574 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.607539    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:35.708168 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.708134    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:35.930971 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.930884    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:35.931570 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:35.931550    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:36.259670 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:36.259633    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:36.259847 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:36.259790    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:36.259911 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:36.259850    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:36.259985 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:36.259961    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:36.485377 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:36.485239    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:11:36.486680 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:36.485687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerStarted","Data":"691d8dea592003b5d591f22ed996a1cd3f4a51f3a2989ff4667df9449ec49d57"}
Sep 06 19:11:36.488521 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:36.488492    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx" event={"ID":"2a3f97c5-5025-4cd1-84e4-bc2f9f406918","Type":"ContainerStarted","Data":"0da26c4f361cb7c41664c597003cff028280a23531114557b41ae1ee68b2d707"}
Sep 06 19:11:36.488917 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:36.488897    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:36.489492 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:36.489470    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-97wm4"
Sep 06 19:11:36.503666 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:36.503617    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hb6hx" podStartSLOduration=3.212738983 podStartE2EDuration="24.503597724s" totalImagesPullingTime="21.290858741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:36.50305423 +0000 UTC m=+24.783507024" watchObservedRunningTime="2026-09-06 19:11:36.503597724 +0000 UTC m=+24.784050520"
Sep 06 19:11:36.561973 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:36.561944    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:37.260136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:37.259921    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:37.260332 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:37.260256    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:38.259501 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:38.259420    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:38.259501 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:38.259467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:38.260009 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:38.259571    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:38.260009 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:38.259682    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:39.259917 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.259745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:39.260296 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:39.260005    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:39.293102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.293051    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:39.354744 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.354671    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:39.498438 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.498402    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:11:39.499295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.499180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerStarted","Data":"9b7cd7b3baa2e3cba610204c85e623d552846982bb7c23bed88429ef9b862b1f"}
Sep 06 19:11:39.501948 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.500628    2581 scope.go:117] "RemoveContainer" containerID="9afef782b28f252b82b189fef02cc37279b45ece9b85d9c42f2b2a0329a504e0"
Sep 06 19:11:39.501948 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.501766    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:39.501948 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.501856    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:39.502181 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.501948    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:39.504439 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.504398    2581 generic.go:356] "Generic (PLEG): container finished" podID="88928f30-fd24-43b0-9f56-3eb2192a0784" containerID="58a5fa343051d71a724f581f15e2eca7b241dfc2c3294d2501dae7ffc5c7a8c9" exitCode=0
Sep 06 19:11:39.504439 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.504428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb7wn" event={"ID":"88928f30-fd24-43b0-9f56-3eb2192a0784","Type":"ContainerDied","Data":"58a5fa343051d71a724f581f15e2eca7b241dfc2c3294d2501dae7ffc5c7a8c9"}
Sep 06 19:11:39.518614 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.518594    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:39.519658 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:39.519642    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:11:40.260289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.260257    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:40.260289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.260294    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:40.260916 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:40.260377    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:40.260916 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:40.260511    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:40.511614 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.511556    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:11:40.511995 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.511966    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" event={"ID":"da3d32c4-7c34-492c-9d9e-70f03708d30d","Type":"ContainerStarted","Data":"e52b1229767bbb9dfda0f458f460575365b1dc2568445058f4144a80877f4815"}
Sep 06 19:11:40.537854 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.537802    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj" podStartSLOduration=9.484255615 podStartE2EDuration="28.537780731s" totalImagesPullingTime="19.053525116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:40.536703816 +0000 UTC m=+28.817156617" watchObservedRunningTime="2026-09-06 19:11:40.537780731 +0000 UTC m=+28.818233558"
Sep 06 19:11:40.761218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.761170    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7djzc"]
Sep 06 19:11:40.761408 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.761351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:40.761495 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:40.761468    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:40.763816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.763754    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xm5z7"]
Sep 06 19:11:40.763912 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.763856    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:40.763950 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:40.763935    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:40.772083 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.772035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jt8wx"]
Sep 06 19:11:40.772221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:40.772190    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:40.772296 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:40.772279    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:41.516440 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:41.516404    2581 generic.go:356] "Generic (PLEG): container finished" podID="88928f30-fd24-43b0-9f56-3eb2192a0784" containerID="a8d0b0682b339713c82d7485904961886419dc1f8a6bb1073b9b5c2b67f53ff2" exitCode=0
Sep 06 19:11:41.517275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:41.516494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb7wn" event={"ID":"88928f30-fd24-43b0-9f56-3eb2192a0784","Type":"ContainerDied","Data":"a8d0b0682b339713c82d7485904961886419dc1f8a6bb1073b9b5c2b67f53ff2"}
Sep 06 19:11:41.617670 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:41.617640    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:41.713861 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:41.713824    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:42.260164 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:42.259976    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:42.260359 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:42.260223    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:42.260359 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:42.260051    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:42.260359 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:42.260295    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:42.260359 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:42.260038    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:42.260562 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:42.260370    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:42.838089 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:42.838022    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:42.889841 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:42.889811    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:43.524210 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:43.524175    2581 generic.go:356] "Generic (PLEG): container finished" podID="88928f30-fd24-43b0-9f56-3eb2192a0784" containerID="7a5e35b999270dd0cdacbec7cd9b7587b3dbffc79ce268afa008a7fc7b269fc5" exitCode=0
Sep 06 19:11:43.524382 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:43.524237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb7wn" event={"ID":"88928f30-fd24-43b0-9f56-3eb2192a0784","Type":"ContainerDied","Data":"7a5e35b999270dd0cdacbec7cd9b7587b3dbffc79ce268afa008a7fc7b269fc5"}
Sep 06 19:11:44.260308 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:44.260276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:44.260308 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:44.260315    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:44.260824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:44.260274    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:44.260824 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:44.260400    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:44.260824 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:44.260504    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:44.260824 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:44.260610    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:44.583711 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:44.583628    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:44.635323 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:44.635287    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:44.951047 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:44.951008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:45.923840 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:45.923799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:45.923840 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:45.923842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:45.924403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:45.923949    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:45.924403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:45.923962    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 19:11:45.924403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:45.923980    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 19:11:45.924403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:45.923992    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kdswv for pod openshift-network-diagnostics/network-check-target-jt8wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:45.924403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:45.924016    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs podName:88608533-3551-4eaf-8716-5757cd3108c0 nodeName:}" failed. No retries permitted until 2026-09-06 19:12:17.923998369 +0000 UTC m=+66.204451160 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs") pod "network-metrics-daemon-xm5z7" (UID: "88608533-3551-4eaf-8716-5757cd3108c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 19:11:45.924403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:45.924033    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv podName:bfc0b5e6-446a-482f-8ea3-087c734a417a nodeName:}" failed. No retries permitted until 2026-09-06 19:12:17.924023492 +0000 UTC m=+66.204476266 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kdswv" (UniqueName: "kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv") pod "network-check-target-jt8wx" (UID: "bfc0b5e6-446a-482f-8ea3-087c734a417a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 19:11:46.025133 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:46.025095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:46.025326 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:46.025235    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:46.025326 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:46.025295    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret podName:5d5de6d9-7cef-4edf-8164-dbb60b3089e9 nodeName:}" failed. No retries permitted until 2026-09-06 19:12:18.02528008 +0000 UTC m=+66.305732872 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret") pod "global-pull-secret-syncer-7djzc" (UID: "5d5de6d9-7cef-4edf-8164-dbb60b3089e9") : object "kube-system"/"original-pull-secret" not registered
Sep 06 19:11:46.260189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:46.260099    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:46.260189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:46.260134    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:46.260189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:46.260134    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:46.260431 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:46.260228    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7djzc" podUID="5d5de6d9-7cef-4edf-8164-dbb60b3089e9"
Sep 06 19:11:46.260431 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:46.260324    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt8wx" podUID="bfc0b5e6-446a-482f-8ea3-087c734a417a"
Sep 06 19:11:46.260431 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:46.260383    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:11:47.003414 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.003383    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-200.ec2.internal" event="NodeReady"
Sep 06 19:11:47.003855 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.003511    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 06 19:11:47.045036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.045004    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7xc4f"]
Sep 06 19:11:47.050753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.050727    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kvr6h"]
Sep 06 19:11:47.050888 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.050851    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.052923 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.052900    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nxfsd\""
Sep 06 19:11:47.053036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.052962    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 06 19:11:47.053036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.052972    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 06 19:11:47.054522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.054499    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kvr6h"
Sep 06 19:11:47.056297 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.056277    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xc4f"]
Sep 06 19:11:47.056954 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.056936    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 06 19:11:47.057655 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.057187    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 06 19:11:47.057655 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.057361    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qk5sj\""
Sep 06 19:11:47.057655 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.057539    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 06 19:11:47.060537 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.060155    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kvr6h"]
Sep 06 19:11:47.133233 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.133197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.133414 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.133254    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvklp\" (UniqueName: \"kubernetes.io/projected/cce4e1bc-30ea-4560-813d-c7c07a1f368e-kube-api-access-pvklp\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.133414 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.133301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmpp7\" (UniqueName: \"kubernetes.io/projected/687926f7-d477-46b8-9142-c4a64f35d6e4-kube-api-access-qmpp7\") pod \"ingress-canary-kvr6h\" (UID: \"687926f7-d477-46b8-9142-c4a64f35d6e4\") " pod="openshift-ingress-canary/ingress-canary-kvr6h"
Sep 06 19:11:47.133414 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.133374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/687926f7-d477-46b8-9142-c4a64f35d6e4-cert\") pod \"ingress-canary-kvr6h\" (UID: \"687926f7-d477-46b8-9142-c4a64f35d6e4\") " pod="openshift-ingress-canary/ingress-canary-kvr6h"
Sep 06 19:11:47.133575 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.133476    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cce4e1bc-30ea-4560-813d-c7c07a1f368e-config-volume\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.133575 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.133503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cce4e1bc-30ea-4560-813d-c7c07a1f368e-tmp-dir\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.155646 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.155605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:47.234570 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.234534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmpp7\" (UniqueName: \"kubernetes.io/projected/687926f7-d477-46b8-9142-c4a64f35d6e4-kube-api-access-qmpp7\") pod \"ingress-canary-kvr6h\" (UID: \"687926f7-d477-46b8-9142-c4a64f35d6e4\") " pod="openshift-ingress-canary/ingress-canary-kvr6h"
Sep 06 19:11:47.234753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.234622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/687926f7-d477-46b8-9142-c4a64f35d6e4-cert\") pod \"ingress-canary-kvr6h\" (UID: \"687926f7-d477-46b8-9142-c4a64f35d6e4\") " pod="openshift-ingress-canary/ingress-canary-kvr6h"
Sep 06 19:11:47.234753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.234658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cce4e1bc-30ea-4560-813d-c7c07a1f368e-config-volume\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.234753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.234680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cce4e1bc-30ea-4560-813d-c7c07a1f368e-tmp-dir\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.234753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.234719    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.234753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.234752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvklp\" (UniqueName: \"kubernetes.io/projected/cce4e1bc-30ea-4560-813d-c7c07a1f368e-kube-api-access-pvklp\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.235102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.235085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cce4e1bc-30ea-4560-813d-c7c07a1f368e-tmp-dir\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.235158 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:47.235092    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 19:11:47.235210 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:47.235157    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls podName:cce4e1bc-30ea-4560-813d-c7c07a1f368e nodeName:}" failed. No retries permitted until 2026-09-06 19:11:47.735140967 +0000 UTC m=+36.015593738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls") pod "dns-default-7xc4f" (UID: "cce4e1bc-30ea-4560-813d-c7c07a1f368e") : secret "dns-default-metrics-tls" not found
Sep 06 19:11:47.235340 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.235299    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cce4e1bc-30ea-4560-813d-c7c07a1f368e-config-volume\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.240043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.240018    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/687926f7-d477-46b8-9142-c4a64f35d6e4-cert\") pod \"ingress-canary-kvr6h\" (UID: \"687926f7-d477-46b8-9142-c4a64f35d6e4\") " pod="openshift-ingress-canary/ingress-canary-kvr6h"
Sep 06 19:11:47.245622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.245600    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvklp\" (UniqueName: \"kubernetes.io/projected/cce4e1bc-30ea-4560-813d-c7c07a1f368e-kube-api-access-pvklp\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.246520 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.246482    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmpp7\" (UniqueName: \"kubernetes.io/projected/687926f7-d477-46b8-9142-c4a64f35d6e4-kube-api-access-qmpp7\") pod \"ingress-canary-kvr6h\" (UID: \"687926f7-d477-46b8-9142-c4a64f35d6e4\") " pod="openshift-ingress-canary/ingress-canary-kvr6h"
Sep 06 19:11:47.256306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.256233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:47.371910 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.371871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kvr6h"
Sep 06 19:11:47.705095 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.704841    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kvr6h"]
Sep 06 19:11:47.730977 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:11:47.730947    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod687926f7_d477_46b8_9142_c4a64f35d6e4.slice/crio-c6e3b01bd370482e0776e6acb1ba00ed3455f8b7887d0a5daecbdf9dbf15ec6e WatchSource:0}: Error finding container c6e3b01bd370482e0776e6acb1ba00ed3455f8b7887d0a5daecbdf9dbf15ec6e: Status 404 returned error can't find the container with id c6e3b01bd370482e0776e6acb1ba00ed3455f8b7887d0a5daecbdf9dbf15ec6e
Sep 06 19:11:47.739304 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:47.739281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:47.739408 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:47.739395    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 19:11:47.739462 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:47.739452    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls podName:cce4e1bc-30ea-4560-813d-c7c07a1f368e nodeName:}" failed. No retries permitted until 2026-09-06 19:11:48.739433926 +0000 UTC m=+37.019886697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls") pod "dns-default-7xc4f" (UID: "cce4e1bc-30ea-4560-813d-c7c07a1f368e") : secret "dns-default-metrics-tls" not found
Sep 06 19:11:48.260392 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.260357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:11:48.260941 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.260357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:11:48.260941 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.260357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:11:48.262921 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.262898    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 19:11:48.263042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.262993    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 19:11:48.263883 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.263864    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 19:11:48.263990 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.263941    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6p7v7\""
Sep 06 19:11:48.263990 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.263960    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-57svr\""
Sep 06 19:11:48.264138 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.264105    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 19:11:48.541216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.541133    2581 generic.go:356] "Generic (PLEG): container finished" podID="88928f30-fd24-43b0-9f56-3eb2192a0784" containerID="ad239f71266eb33b1ddee081feb233077db99d64f30f65839daa84bf74e8b56e" exitCode=0
Sep 06 19:11:48.541373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.541227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb7wn" event={"ID":"88928f30-fd24-43b0-9f56-3eb2192a0784","Type":"ContainerDied","Data":"ad239f71266eb33b1ddee081feb233077db99d64f30f65839daa84bf74e8b56e"}
Sep 06 19:11:48.542473 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.542446    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kvr6h" event={"ID":"687926f7-d477-46b8-9142-c4a64f35d6e4","Type":"ContainerStarted","Data":"c6e3b01bd370482e0776e6acb1ba00ed3455f8b7887d0a5daecbdf9dbf15ec6e"}
Sep 06 19:11:48.745908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:48.745870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:48.746107 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:48.746055    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 19:11:48.746159 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:48.746151    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls podName:cce4e1bc-30ea-4560-813d-c7c07a1f368e nodeName:}" failed. No retries permitted until 2026-09-06 19:11:50.746129241 +0000 UTC m=+39.026582020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls") pod "dns-default-7xc4f" (UID: "cce4e1bc-30ea-4560-813d-c7c07a1f368e") : secret "dns-default-metrics-tls" not found
Sep 06 19:11:49.548860 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:49.548825    2581 generic.go:356] "Generic (PLEG): container finished" podID="88928f30-fd24-43b0-9f56-3eb2192a0784" containerID="605752269dbe7e46e8527b6a452e5c5c7188b01c55f6a588321cf68ff70537f7" exitCode=0
Sep 06 19:11:49.549563 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:49.548909    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb7wn" event={"ID":"88928f30-fd24-43b0-9f56-3eb2192a0784","Type":"ContainerDied","Data":"605752269dbe7e46e8527b6a452e5c5c7188b01c55f6a588321cf68ff70537f7"}
Sep 06 19:11:50.555124 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:50.555097    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb7wn" event={"ID":"88928f30-fd24-43b0-9f56-3eb2192a0784","Type":"ContainerStarted","Data":"a16759f758822fb863c3ebe911a857f97e13e9cd6bae5148e6cca8293130196d"}
Sep 06 19:11:50.573183 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:50.573136    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gb7wn" podStartSLOduration=8.748307386 podStartE2EDuration="38.573121153s" totalImagesPullingTime="29.824813767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:50.572120885 +0000 UTC m=+38.852573678" watchObservedRunningTime="2026-09-06 19:11:50.573121153 +0000 UTC m=+38.853573946"
Sep 06 19:11:50.692988 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:50.692882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:50.745635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:50.745602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:50.761674 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:50.761643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:50.761775 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:50.761763    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 19:11:50.761824 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:50.761815    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls podName:cce4e1bc-30ea-4560-813d-c7c07a1f368e nodeName:}" failed. No retries permitted until 2026-09-06 19:11:54.761802879 +0000 UTC m=+43.042255650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls") pod "dns-default-7xc4f" (UID: "cce4e1bc-30ea-4560-813d-c7c07a1f368e") : secret "dns-default-metrics-tls" not found
Sep 06 19:11:51.185699 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:51.185481    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:51.285993 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:51.285946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:51.563932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:51.563829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kvr6h" event={"ID":"687926f7-d477-46b8-9142-c4a64f35d6e4","Type":"ContainerStarted","Data":"7edc6291c28a82760fba6c20a5ebac59cb2530ea8e3c853b4f908e728e800eb1"}
Sep 06 19:11:51.579241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:51.579192    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kvr6h" podStartSLOduration=1.894690532 podStartE2EDuration="4.579177409s" totalImagesPullingTime="2.684486877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:11:51.578007252 +0000 UTC m=+39.858460047" watchObservedRunningTime="2026-09-06 19:11:51.579177409 +0000 UTC m=+39.859630202"
Sep 06 19:11:54.776901 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:54.776867    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:11:54.789962 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:54.789935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:11:54.790055 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:54.790046    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 19:11:54.790137 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:11:54.790127    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls podName:cce4e1bc-30ea-4560-813d-c7c07a1f368e nodeName:}" failed. No retries permitted until 2026-09-06 19:12:02.790112754 +0000 UTC m=+51.070565526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls") pod "dns-default-7xc4f" (UID: "cce4e1bc-30ea-4560-813d-c7c07a1f368e") : secret "dns-default-metrics-tls" not found
Sep 06 19:11:54.877296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:11:54.877259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:12:02.841451 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:02.841404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:12:02.841901 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:12:02.841528    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 19:12:02.841901 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:12:02.841587    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls podName:cce4e1bc-30ea-4560-813d-c7c07a1f368e nodeName:}" failed. No retries permitted until 2026-09-06 19:12:18.841572676 +0000 UTC m=+67.122025449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls") pod "dns-default-7xc4f" (UID: "cce4e1bc-30ea-4560-813d-c7c07a1f368e") : secret "dns-default-metrics-tls" not found
Sep 06 19:12:05.611548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:05.611510    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:12:05.713605 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:05.713576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:12:11.539263 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:11.539232    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4frj"
Sep 06 19:12:11.643748 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:11.643705    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:12:11.699589 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:11.699557    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:12:17.937104 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:17.937035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:12:17.937688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:17.937124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:12:17.939951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:17.939927    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 19:12:17.940089 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:17.940048    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 19:12:17.947623 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:12:17.947597    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 19:12:17.947714 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:12:17.947683    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs podName:88608533-3551-4eaf-8716-5757cd3108c0 nodeName:}" failed. No retries permitted until 2026-09-06 19:13:21.947664968 +0000 UTC m=+130.228117739 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs") pod "network-metrics-daemon-xm5z7" (UID: "88608533-3551-4eaf-8716-5757cd3108c0") : secret "metrics-daemon-secret" not found
Sep 06 19:12:17.949469 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:17.949452    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 19:12:17.961347 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:17.961326    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdswv\" (UniqueName: \"kubernetes.io/projected/bfc0b5e6-446a-482f-8ea3-087c734a417a-kube-api-access-kdswv\") pod \"network-check-target-jt8wx\" (UID: \"bfc0b5e6-446a-482f-8ea3-087c734a417a\") " pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:12:17.982449 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:17.982413    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-57svr\""
Sep 06 19:12:17.990981 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:17.990948    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:12:18.037475 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:18.037438    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:12:18.039569 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:18.039543    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 19:12:18.050740 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:18.050707    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d5de6d9-7cef-4edf-8164-dbb60b3089e9-original-pull-secret\") pod \"global-pull-secret-syncer-7djzc\" (UID: \"5d5de6d9-7cef-4edf-8164-dbb60b3089e9\") " pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:12:18.263753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:18.263725    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jt8wx"]
Sep 06 19:12:18.264730 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:12:18.264705    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc0b5e6_446a_482f_8ea3_087c734a417a.slice/crio-4ed4ffd15aa2996a80e2feddc4cc06362abfbb629dd48428f9bc158b0ed196e8 WatchSource:0}: Error finding container 4ed4ffd15aa2996a80e2feddc4cc06362abfbb629dd48428f9bc158b0ed196e8: Status 404 returned error can't find the container with id 4ed4ffd15aa2996a80e2feddc4cc06362abfbb629dd48428f9bc158b0ed196e8
Sep 06 19:12:18.286898 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:18.286868    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7djzc"
Sep 06 19:12:18.592835 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:18.592752    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7djzc"]
Sep 06 19:12:18.597532 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:12:18.597508    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5de6d9_7cef_4edf_8164_dbb60b3089e9.slice/crio-122b2a4e11718ba6b268d19e0ac16c7c946ef693431b9ca1e94b17a72f01cee3 WatchSource:0}: Error finding container 122b2a4e11718ba6b268d19e0ac16c7c946ef693431b9ca1e94b17a72f01cee3: Status 404 returned error can't find the container with id 122b2a4e11718ba6b268d19e0ac16c7c946ef693431b9ca1e94b17a72f01cee3
Sep 06 19:12:18.631886 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:18.631848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7djzc" event={"ID":"5d5de6d9-7cef-4edf-8164-dbb60b3089e9","Type":"ContainerStarted","Data":"122b2a4e11718ba6b268d19e0ac16c7c946ef693431b9ca1e94b17a72f01cee3"}
Sep 06 19:12:18.633159 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:18.633128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jt8wx" event={"ID":"bfc0b5e6-446a-482f-8ea3-087c734a417a","Type":"ContainerStarted","Data":"4ed4ffd15aa2996a80e2feddc4cc06362abfbb629dd48428f9bc158b0ed196e8"}
Sep 06 19:12:18.843268 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:18.843175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:12:18.843422 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:12:18.843339    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 19:12:18.843422 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:12:18.843419    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls podName:cce4e1bc-30ea-4560-813d-c7c07a1f368e nodeName:}" failed. No retries permitted until 2026-09-06 19:12:50.843395735 +0000 UTC m=+99.123848510 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls") pod "dns-default-7xc4f" (UID: "cce4e1bc-30ea-4560-813d-c7c07a1f368e") : secret "dns-default-metrics-tls" not found
Sep 06 19:12:23.650995 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:23.650954    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jt8wx" event={"ID":"bfc0b5e6-446a-482f-8ea3-087c734a417a","Type":"ContainerStarted","Data":"9ec134448f6d2d5acd29298c3d6c7a087291234d7190ee37abb794f79b6c093c"}
Sep 06 19:12:23.651502 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:23.651038    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:12:23.652401 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:23.652379    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7djzc" event={"ID":"5d5de6d9-7cef-4edf-8164-dbb60b3089e9","Type":"ContainerStarted","Data":"b7d46acac9da41978644b920701e644e954a27aa3112a0351647f525ef351317"}
Sep 06 19:12:23.663976 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:23.663919    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jt8wx" podStartSLOduration=66.906790493 podStartE2EDuration="1m11.663900786s" totalImagesPullingTime="4.757110293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:12:23.66349041 +0000 UTC m=+71.943943214" watchObservedRunningTime="2026-09-06 19:12:23.663900786 +0000 UTC m=+71.944353571"
Sep 06 19:12:23.675560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:23.675516    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7djzc" podStartSLOduration=67.246504898 podStartE2EDuration="1m11.675499556s" totalImagesPullingTime="4.428994658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:12:23.675288862 +0000 UTC m=+71.955741667" watchObservedRunningTime="2026-09-06 19:12:23.675499556 +0000 UTC m=+71.955952349"
Sep 06 19:12:50.862088 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:50.862020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:12:50.862490 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:12:50.862186    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 19:12:50.862490 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:12:50.862256    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls podName:cce4e1bc-30ea-4560-813d-c7c07a1f368e nodeName:}" failed. No retries permitted until 2026-09-06 19:13:54.862239638 +0000 UTC m=+163.142692410 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls") pod "dns-default-7xc4f" (UID: "cce4e1bc-30ea-4560-813d-c7c07a1f368e") : secret "dns-default-metrics-tls" not found
Sep 06 19:12:54.657630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:12:54.657593    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jt8wx"
Sep 06 19:13:21.969854 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:13:21.969813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:13:21.970324 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:13:21.969976    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 19:13:21.970324 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:13:21.970057    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs podName:88608533-3551-4eaf-8716-5757cd3108c0 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:23.970039243 +0000 UTC m=+252.250492015 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs") pod "network-metrics-daemon-xm5z7" (UID: "88608533-3551-4eaf-8716-5757cd3108c0") : secret "metrics-daemon-secret" not found
Sep 06 19:13:50.065601 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:13:50.065537    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7xc4f" podUID="cce4e1bc-30ea-4560-813d-c7c07a1f368e"
Sep 06 19:13:50.870045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:13:50.870014    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:13:51.273283 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:13:51.273234    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xm5z7" podUID="88608533-3551-4eaf-8716-5757cd3108c0"
Sep 06 19:13:54.884910 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:13:54.884866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:13:54.885344 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:13:54.885020    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 19:13:54.885344 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:13:54.885126    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls podName:cce4e1bc-30ea-4560-813d-c7c07a1f368e nodeName:}" failed. No retries permitted until 2026-09-06 19:15:56.885107949 +0000 UTC m=+285.165560720 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls") pod "dns-default-7xc4f" (UID: "cce4e1bc-30ea-4560-813d-c7c07a1f368e") : secret "dns-default-metrics-tls" not found
Sep 06 19:14:03.260325 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:03.260232    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:14:36.748679 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:36.748647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:14:59.951531 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.951482    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-sz7c7"]
Sep 06 19:14:59.954778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.954754    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-sz7c7"
Sep 06 19:14:59.956810 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.956792    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wmjxx\""
Sep 06 19:14:59.963925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.963898    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-576f88fb4d-zrv9b"]
Sep 06 19:14:59.966600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.966577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:14:59.967164 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.967144    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-sz7c7"]
Sep 06 19:14:59.968460 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.968433    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 06 19:14:59.968561 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.968464    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 06 19:14:59.968561 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.968488    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 06 19:14:59.968787 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.968775    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 06 19:14:59.969185 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.969168    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 06 19:14:59.969247 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.969222    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8clg5\""
Sep 06 19:14:59.969299 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.969268    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 06 19:14:59.980695 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:14:59.980668    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-576f88fb4d-zrv9b"]
Sep 06 19:15:00.011141 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.011105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.011318 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.011169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-default-certificate\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.011318 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.011208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-stats-auth\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.011318 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.011251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq8kx\" (UniqueName: \"kubernetes.io/projected/70c0a808-8fad-4a11-bcd1-4c79e8353373-kube-api-access-xq8kx\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.011318 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.011315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.011478 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.011335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5sdg\" (UniqueName: \"kubernetes.io/projected/78c82aef-971e-4211-8dae-09a0a0ea3979-kube-api-access-w5sdg\") pod \"network-check-source-685456655f-sz7c7\" (UID: \"78c82aef-971e-4211-8dae-09a0a0ea3979\") " pod="openshift-network-diagnostics/network-check-source-685456655f-sz7c7"
Sep 06 19:15:00.047280 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.047251    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"]
Sep 06 19:15:00.050005 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.049989    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.052059 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.052031    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 06 19:15:00.052200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.052100    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 06 19:15:00.052200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.052163    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 06 19:15:00.052200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.052184    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 06 19:15:00.052363 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.052320    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zrbgw\""
Sep 06 19:15:00.054084 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.054026    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"]
Sep 06 19:15:00.056913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.056893    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-ttcnn"]
Sep 06 19:15:00.057247 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.057019    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.059211 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.059192    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 06 19:15:00.059315 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.059200    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8vx46\""
Sep 06 19:15:00.059374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.059340    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 06 19:15:00.059483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.059460    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 06 19:15:00.059594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.059498    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 06 19:15:00.059975 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.059946    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"]
Sep 06 19:15:00.060095 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.060044    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.061747 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.061725    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 06 19:15:00.062001 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.061986    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 06 19:15:00.062100 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.061989    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tdlcv\""
Sep 06 19:15:00.062190 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.062172    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 06 19:15:00.062255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.062203    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 06 19:15:00.067901 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.067880    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 06 19:15:00.068349 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.068328    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-ttcnn"]
Sep 06 19:15:00.071697 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.071675    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"]
Sep 06 19:15:00.112114 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.112114 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112115    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e97e47da-6bc6-48f8-bf89-18da9c037980-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-w6lgm\" (UID: \"e97e47da-6bc6-48f8-bf89-18da9c037980\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e97e47da-6bc6-48f8-bf89-18da9c037980-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-w6lgm\" (UID: \"e97e47da-6bc6-48f8-bf89-18da9c037980\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5sdg\" (UniqueName: \"kubernetes.io/projected/78c82aef-971e-4211-8dae-09a0a0ea3979-kube-api-access-w5sdg\") pod \"network-check-source-685456655f-sz7c7\" (UID: \"78c82aef-971e-4211-8dae-09a0a0ea3979\") " pod="openshift-network-diagnostics/network-check-source-685456655f-sz7c7"
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/869540bb-a576-4edf-9c8c-0aaa8c64ccc9-config\") pod \"service-ca-operator-859f94c4b-vnqh6\" (UID: \"869540bb-a576-4edf-9c8c-0aaa8c64ccc9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:00.112197    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:00.112270    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:00.61225371 +0000 UTC m=+228.892706482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : secret "router-metrics-certs-default" not found
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112293    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-serving-cert\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-trusted-ca\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ch98\" (UniqueName: \"kubernetes.io/projected/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-kube-api-access-2ch98\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.112343 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:00.112350    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:00.61233371 +0000 UTC m=+228.892786482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : configmap references non-existent config key: service-ca.crt
Sep 06 19:15:00.112731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112390    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kznc5\" (UniqueName: \"kubernetes.io/projected/e97e47da-6bc6-48f8-bf89-18da9c037980-kube-api-access-kznc5\") pod \"kube-storage-version-migrator-operator-8d7dd4675-w6lgm\" (UID: \"e97e47da-6bc6-48f8-bf89-18da9c037980\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.112731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112412    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/869540bb-a576-4edf-9c8c-0aaa8c64ccc9-serving-cert\") pod \"service-ca-operator-859f94c4b-vnqh6\" (UID: \"869540bb-a576-4edf-9c8c-0aaa8c64ccc9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.112731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-config\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.112731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-default-certificate\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.112731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msd5r\" (UniqueName: \"kubernetes.io/projected/869540bb-a576-4edf-9c8c-0aaa8c64ccc9-kube-api-access-msd5r\") pod \"service-ca-operator-859f94c4b-vnqh6\" (UID: \"869540bb-a576-4edf-9c8c-0aaa8c64ccc9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.112731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-stats-auth\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.112731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.112523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq8kx\" (UniqueName: \"kubernetes.io/projected/70c0a808-8fad-4a11-bcd1-4c79e8353373-kube-api-access-xq8kx\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.115002 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.114975    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-default-certificate\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.115122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.115030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-stats-auth\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.121650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.121628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq8kx\" (UniqueName: \"kubernetes.io/projected/70c0a808-8fad-4a11-bcd1-4c79e8353373-kube-api-access-xq8kx\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.121751 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.121654    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5sdg\" (UniqueName: \"kubernetes.io/projected/78c82aef-971e-4211-8dae-09a0a0ea3979-kube-api-access-w5sdg\") pod \"network-check-source-685456655f-sz7c7\" (UID: \"78c82aef-971e-4211-8dae-09a0a0ea3979\") " pod="openshift-network-diagnostics/network-check-source-685456655f-sz7c7"
Sep 06 19:15:00.172680 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.172643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:00.213512 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e97e47da-6bc6-48f8-bf89-18da9c037980-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-w6lgm\" (UID: \"e97e47da-6bc6-48f8-bf89-18da9c037980\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.213512 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e97e47da-6bc6-48f8-bf89-18da9c037980-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-w6lgm\" (UID: \"e97e47da-6bc6-48f8-bf89-18da9c037980\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.213512 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/869540bb-a576-4edf-9c8c-0aaa8c64ccc9-config\") pod \"service-ca-operator-859f94c4b-vnqh6\" (UID: \"869540bb-a576-4edf-9c8c-0aaa8c64ccc9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.213772 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-serving-cert\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.213772 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-trusted-ca\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.213772 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213637    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ch98\" (UniqueName: \"kubernetes.io/projected/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-kube-api-access-2ch98\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.213772 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kznc5\" (UniqueName: \"kubernetes.io/projected/e97e47da-6bc6-48f8-bf89-18da9c037980-kube-api-access-kznc5\") pod \"kube-storage-version-migrator-operator-8d7dd4675-w6lgm\" (UID: \"e97e47da-6bc6-48f8-bf89-18da9c037980\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.213772 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/869540bb-a576-4edf-9c8c-0aaa8c64ccc9-serving-cert\") pod \"service-ca-operator-859f94c4b-vnqh6\" (UID: \"869540bb-a576-4edf-9c8c-0aaa8c64ccc9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.213772 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-config\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.213772 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.213765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msd5r\" (UniqueName: \"kubernetes.io/projected/869540bb-a576-4edf-9c8c-0aaa8c64ccc9-kube-api-access-msd5r\") pod \"service-ca-operator-859f94c4b-vnqh6\" (UID: \"869540bb-a576-4edf-9c8c-0aaa8c64ccc9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.214276 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.214203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/869540bb-a576-4edf-9c8c-0aaa8c64ccc9-config\") pod \"service-ca-operator-859f94c4b-vnqh6\" (UID: \"869540bb-a576-4edf-9c8c-0aaa8c64ccc9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.214544 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.214510    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e97e47da-6bc6-48f8-bf89-18da9c037980-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-w6lgm\" (UID: \"e97e47da-6bc6-48f8-bf89-18da9c037980\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.214637 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.214548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-trusted-ca\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.215027 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.215007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-config\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.215916 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.215894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e97e47da-6bc6-48f8-bf89-18da9c037980-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-w6lgm\" (UID: \"e97e47da-6bc6-48f8-bf89-18da9c037980\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.216144 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.216124    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/869540bb-a576-4edf-9c8c-0aaa8c64ccc9-serving-cert\") pod \"service-ca-operator-859f94c4b-vnqh6\" (UID: \"869540bb-a576-4edf-9c8c-0aaa8c64ccc9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.216246 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.216232    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-serving-cert\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.220789 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.220766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kznc5\" (UniqueName: \"kubernetes.io/projected/e97e47da-6bc6-48f8-bf89-18da9c037980-kube-api-access-kznc5\") pod \"kube-storage-version-migrator-operator-8d7dd4675-w6lgm\" (UID: \"e97e47da-6bc6-48f8-bf89-18da9c037980\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.221004 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.220972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msd5r\" (UniqueName: \"kubernetes.io/projected/869540bb-a576-4edf-9c8c-0aaa8c64ccc9-kube-api-access-msd5r\") pod \"service-ca-operator-859f94c4b-vnqh6\" (UID: \"869540bb-a576-4edf-9c8c-0aaa8c64ccc9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.221205 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.221179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ch98\" (UniqueName: \"kubernetes.io/projected/3e3eadb1-92f7-4a9f-aa67-eab19946cbc0-kube-api-access-2ch98\") pod \"console-operator-85b454874b-ttcnn\" (UID: \"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0\") " pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.263821 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.263790    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-sz7c7"
Sep 06 19:15:00.273757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.273721    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:00.361658 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.361612    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"
Sep 06 19:15:00.369805 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.369781    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"
Sep 06 19:15:00.375746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.375721    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:00.530727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.530645    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-sz7c7"]
Sep 06 19:15:00.534197 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:00.534171    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c82aef_971e_4211_8dae_09a0a0ea3979.slice/crio-9c1c05adc06da3a5a9c4c963eaa849beb1f4260bf741943aa7ba2ccca301ba1a WatchSource:0}: Error finding container 9c1c05adc06da3a5a9c4c963eaa849beb1f4260bf741943aa7ba2ccca301ba1a: Status 404 returned error can't find the container with id 9c1c05adc06da3a5a9c4c963eaa849beb1f4260bf741943aa7ba2ccca301ba1a
Sep 06 19:15:00.616257 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.616227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.616415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.616397    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:00.616500 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:00.616486    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:01.616464837 +0000 UTC m=+229.896917609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : configmap references non-existent config key: service-ca.crt
Sep 06 19:15:00.616590 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:00.616529    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 19:15:00.616590 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:00.616578    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:01.616564854 +0000 UTC m=+229.897017628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : secret "router-metrics-certs-default" not found
Sep 06 19:15:00.634246 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.634216    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6"]
Sep 06 19:15:00.637358 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:00.637330    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869540bb_a576_4edf_9c8c_0aaa8c64ccc9.slice/crio-2789e337814d4c712e88316af9af32f622febcef29019039f1438568e8800276 WatchSource:0}: Error finding container 2789e337814d4c712e88316af9af32f622febcef29019039f1438568e8800276: Status 404 returned error can't find the container with id 2789e337814d4c712e88316af9af32f622febcef29019039f1438568e8800276
Sep 06 19:15:00.660550 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.660521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm"]
Sep 06 19:15:00.663661 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:00.663634    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-ttcnn"]
Sep 06 19:15:00.664884 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:00.664779    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97e47da_6bc6_48f8_bf89_18da9c037980.slice/crio-3ba07aa436d39d482f1e4a9860c29b0dbf7254915ca256b50c901f9a908a3938 WatchSource:0}: Error finding container 3ba07aa436d39d482f1e4a9860c29b0dbf7254915ca256b50c901f9a908a3938: Status 404 returned error can't find the container with id 3ba07aa436d39d482f1e4a9860c29b0dbf7254915ca256b50c901f9a908a3938
Sep 06 19:15:00.666628 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:00.666603    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3eadb1_92f7_4a9f_aa67_eab19946cbc0.slice/crio-009d6baab0f33ce5879ed6568f058a3f131cd2a94669f894088b8a96f0f9fa62 WatchSource:0}: Error finding container 009d6baab0f33ce5879ed6568f058a3f131cd2a94669f894088b8a96f0f9fa62: Status 404 returned error can't find the container with id 009d6baab0f33ce5879ed6568f058a3f131cd2a94669f894088b8a96f0f9fa62
Sep 06 19:15:01.046481 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:01.046448    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-sz7c7" event={"ID":"78c82aef-971e-4211-8dae-09a0a0ea3979","Type":"ContainerStarted","Data":"429900ccd6917022e1c0e84b8aab1260c93493b579d892130cb12239c999842f"}
Sep 06 19:15:01.046954 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:01.046494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-sz7c7" event={"ID":"78c82aef-971e-4211-8dae-09a0a0ea3979","Type":"ContainerStarted","Data":"9c1c05adc06da3a5a9c4c963eaa849beb1f4260bf741943aa7ba2ccca301ba1a"}
Sep 06 19:15:01.047759 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:01.047737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-ttcnn" event={"ID":"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0","Type":"ContainerStarted","Data":"009d6baab0f33ce5879ed6568f058a3f131cd2a94669f894088b8a96f0f9fa62"}
Sep 06 19:15:01.048773 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:01.048753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6" event={"ID":"869540bb-a576-4edf-9c8c-0aaa8c64ccc9","Type":"ContainerStarted","Data":"2789e337814d4c712e88316af9af32f622febcef29019039f1438568e8800276"}
Sep 06 19:15:01.049921 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:01.049901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm" event={"ID":"e97e47da-6bc6-48f8-bf89-18da9c037980","Type":"ContainerStarted","Data":"3ba07aa436d39d482f1e4a9860c29b0dbf7254915ca256b50c901f9a908a3938"}
Sep 06 19:15:01.060594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:01.060545    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-sz7c7" podStartSLOduration=2.060530581 podStartE2EDuration="2.060530581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:14:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:01.060234831 +0000 UTC m=+229.340687626" watchObservedRunningTime="2026-09-06 19:15:01.060530581 +0000 UTC m=+229.340983376"
Sep 06 19:15:01.626085 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:01.626031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:01.626295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:01.626166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:01.626365 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:01.626317    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 19:15:01.626417 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:01.626388    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:03.626369802 +0000 UTC m=+231.906822581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : secret "router-metrics-certs-default" not found
Sep 06 19:15:01.626833 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:01.626813    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:03.626799234 +0000 UTC m=+231.907252009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : configmap references non-existent config key: service-ca.crt
Sep 06 19:15:03.063254 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.063210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:03.644075 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.644012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:03.644277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.644136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:03.644350 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:03.644285    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 19:15:03.644350 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:03.644336    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:07.644311266 +0000 UTC m=+235.924764037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : configmap references non-existent config key: service-ca.crt
Sep 06 19:15:03.644462 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:03.644371    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:07.644357785 +0000 UTC m=+235.924810571 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : secret "router-metrics-certs-default" not found
Sep 06 19:15:03.750828 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.750793    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g77gb"]
Sep 06 19:15:03.753958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.753941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.755896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.755879    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 06 19:15:03.756017 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.755897    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 06 19:15:03.756294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.756243    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 06 19:15:03.756567 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.756550    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 06 19:15:03.756614 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.756601    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-grwwn\""
Sep 06 19:15:03.757440 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.757420    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:03.764532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.764472    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g77gb"]
Sep 06 19:15:03.814600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.814572    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:03.846396 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.846356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.846396 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.846393    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.846615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.846411    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t2vt\" (UniqueName: \"kubernetes.io/projected/e621832d-7142-4720-a723-066c1cfb5adf-kube-api-access-8t2vt\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.846615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.846454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e621832d-7142-4720-a723-066c1cfb5adf-crio-socket\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.846615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.846526    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e621832d-7142-4720-a723-066c1cfb5adf-data-volume\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.947706 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.947609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e621832d-7142-4720-a723-066c1cfb5adf-data-volume\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.947868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.947706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.947868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.947739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.947868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.947762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t2vt\" (UniqueName: \"kubernetes.io/projected/e621832d-7142-4720-a723-066c1cfb5adf-kube-api-access-8t2vt\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.947868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.947808    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e621832d-7142-4720-a723-066c1cfb5adf-crio-socket\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.947868 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:03.947830    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:03.947868 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:03.947863    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 19:15:03.948176 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.947888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e621832d-7142-4720-a723-066c1cfb5adf-crio-socket\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.948176 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:03.947907    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:04.447885418 +0000 UTC m=+232.728338194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:03.948176 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:03.947932    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:04.44791385 +0000 UTC m=+232.728366634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : configmap "kube-rbac-proxy" not found
Sep 06 19:15:03.948176 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.948005    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e621832d-7142-4720-a723-066c1cfb5adf-data-volume\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:03.955605 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:03.955573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t2vt\" (UniqueName: \"kubernetes.io/projected/e621832d-7142-4720-a723-066c1cfb5adf-kube-api-access-8t2vt\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:04.061099 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.061040    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm" event={"ID":"e97e47da-6bc6-48f8-bf89-18da9c037980","Type":"ContainerStarted","Data":"e6f70f18cf94611867449a0d36c5d0a9d58c642c0c3c3cbec810e78e12a37bdb"}
Sep 06 19:15:04.062859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.062833    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/0.log"
Sep 06 19:15:04.062995 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.062880    2581 generic.go:356] "Generic (PLEG): container finished" podID="3e3eadb1-92f7-4a9f-aa67-eab19946cbc0" containerID="ba1edce74b093995403b96d0ff3d5f6fb4e99d9dbc12c8530b565cf7a1c4c4aa" exitCode=255
Sep 06 19:15:04.062995 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.062962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-ttcnn" event={"ID":"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0","Type":"ContainerDied","Data":"ba1edce74b093995403b96d0ff3d5f6fb4e99d9dbc12c8530b565cf7a1c4c4aa"}
Sep 06 19:15:04.063174 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.063158    2581 scope.go:117] "RemoveContainer" containerID="ba1edce74b093995403b96d0ff3d5f6fb4e99d9dbc12c8530b565cf7a1c4c4aa"
Sep 06 19:15:04.064640 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.064619    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6" event={"ID":"869540bb-a576-4edf-9c8c-0aaa8c64ccc9","Type":"ContainerStarted","Data":"89b57d4cfca1eff096a044632ad158b45ddd60e0358e7e6835d2fb7317479dd2"}
Sep 06 19:15:04.076615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.076572    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm" podStartSLOduration=0.997281987 podStartE2EDuration="4.076556291s" totalImagesPullingTime="3.079274304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:04.075675773 +0000 UTC m=+232.356128577" watchObservedRunningTime="2026-09-06 19:15:04.076556291 +0000 UTC m=+232.357009076"
Sep 06 19:15:04.093744 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.090195    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6" podStartSLOduration=0.978688684 podStartE2EDuration="4.090167975s" totalImagesPullingTime="3.111479291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:04.089271486 +0000 UTC m=+232.369724275" watchObservedRunningTime="2026-09-06 19:15:04.090167975 +0000 UTC m=+232.370620769"
Sep 06 19:15:04.452656 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.452561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:04.452656 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:04.452619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:04.452868 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:04.452711    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:04.452868 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:04.452772    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 19:15:04.452868 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:04.452783    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:05.452766011 +0000 UTC m=+233.733218801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:04.452868 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:04.452862    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:05.452843061 +0000 UTC m=+233.733295840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : configmap "kube-rbac-proxy" not found
Sep 06 19:15:05.069999 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.069969    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:15:05.070487 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.070434    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/0.log"
Sep 06 19:15:05.070487 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.070470    2581 generic.go:356] "Generic (PLEG): container finished" podID="3e3eadb1-92f7-4a9f-aa67-eab19946cbc0" containerID="db2cd57c310de88a24f94da8dd0c706c98ef69cfa18a08755a6bc43ce2127aee" exitCode=255
Sep 06 19:15:05.070595 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.070564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-ttcnn" event={"ID":"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0","Type":"ContainerDied","Data":"db2cd57c310de88a24f94da8dd0c706c98ef69cfa18a08755a6bc43ce2127aee"}
Sep 06 19:15:05.070667 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.070652    2581 scope.go:117] "RemoveContainer" containerID="ba1edce74b093995403b96d0ff3d5f6fb4e99d9dbc12c8530b565cf7a1c4c4aa"
Sep 06 19:15:05.071118 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.070852    2581 scope.go:117] "RemoveContainer" containerID="db2cd57c310de88a24f94da8dd0c706c98ef69cfa18a08755a6bc43ce2127aee"
Sep 06 19:15:05.071118 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:05.071054    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-ttcnn_openshift-console-operator(3e3eadb1-92f7-4a9f-aa67-eab19946cbc0)\"" pod="openshift-console-operator/console-operator-85b454874b-ttcnn" podUID="3e3eadb1-92f7-4a9f-aa67-eab19946cbc0"
Sep 06 19:15:05.279181 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.279145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:05.296898 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.296827    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268"]
Sep 06 19:15:05.301245 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.301218    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268"
Sep 06 19:15:05.303402 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.303374    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 06 19:15:05.303537 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.303376    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-xgpkk\""
Sep 06 19:15:05.303537 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.303435    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 06 19:15:05.308375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.308350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268"]
Sep 06 19:15:05.359875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.359796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcr7w\" (UniqueName: \"kubernetes.io/projected/ca149796-d1f4-4805-a50e-3fce6118c178-kube-api-access-kcr7w\") pod \"migrator-fdbb8fdff-cz268\" (UID: \"ca149796-d1f4-4805-a50e-3fce6118c178\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268"
Sep 06 19:15:05.375478 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.375445    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:05.461032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.460995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:05.461214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.461037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:05.461214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.461119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcr7w\" (UniqueName: \"kubernetes.io/projected/ca149796-d1f4-4805-a50e-3fce6118c178-kube-api-access-kcr7w\") pod \"migrator-fdbb8fdff-cz268\" (UID: \"ca149796-d1f4-4805-a50e-3fce6118c178\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268"
Sep 06 19:15:05.461214 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:05.461187    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 19:15:05.461214 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:05.461189    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:05.461340 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:05.461251    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:07.461233943 +0000 UTC m=+235.741686715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : configmap "kube-rbac-proxy" not found
Sep 06 19:15:05.461340 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:05.461266    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:07.461259802 +0000 UTC m=+235.741712574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:05.468777 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.468746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcr7w\" (UniqueName: \"kubernetes.io/projected/ca149796-d1f4-4805-a50e-3fce6118c178-kube-api-access-kcr7w\") pod \"migrator-fdbb8fdff-cz268\" (UID: \"ca149796-d1f4-4805-a50e-3fce6118c178\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268"
Sep 06 19:15:05.475279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.475241    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:05.577421 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.577387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:05.613436 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.613364    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268"
Sep 06 19:15:05.862072 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.862042    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:05.884325 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.884293    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268"]
Sep 06 19:15:05.888556 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:05.888525    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca149796_d1f4_4805_a50e_3fce6118c178.slice/crio-977916818d03590dfecfc32ee6c2b657e6867bea7963d25fd24be2ae9dac878a WatchSource:0}: Error finding container 977916818d03590dfecfc32ee6c2b657e6867bea7963d25fd24be2ae9dac878a: Status 404 returned error can't find the container with id 977916818d03590dfecfc32ee6c2b657e6867bea7963d25fd24be2ae9dac878a
Sep 06 19:15:05.915227 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.915198    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sf5l2_fae72b4c-8fa1-4c47-a810-601cdffc6722/dns-node-resolver/0.log"
Sep 06 19:15:05.962718 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:05.962689    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:06.075845 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:06.075820    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:15:06.076280 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:06.076194    2581 scope.go:117] "RemoveContainer" containerID="db2cd57c310de88a24f94da8dd0c706c98ef69cfa18a08755a6bc43ce2127aee"
Sep 06 19:15:06.076405 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:06.076385    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-ttcnn_openshift-console-operator(3e3eadb1-92f7-4a9f-aa67-eab19946cbc0)\"" pod="openshift-console-operator/console-operator-85b454874b-ttcnn" podUID="3e3eadb1-92f7-4a9f-aa67-eab19946cbc0"
Sep 06 19:15:06.077106 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:06.077083    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268" event={"ID":"ca149796-d1f4-4805-a50e-3fce6118c178","Type":"ContainerStarted","Data":"977916818d03590dfecfc32ee6c2b657e6867bea7963d25fd24be2ae9dac878a"}
Sep 06 19:15:06.145935 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:06.145846    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:06.246699 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:06.246654    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:06.715636 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:06.715610    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gkckz_9b36afd6-1df8-49b6-ba26-4a7e05a5dd38/node-ca/0.log"
Sep 06 19:15:07.477011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:07.476916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:07.477011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:07.476952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:07.477434 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:07.477037    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 19:15:07.477434 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:07.477093    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:07.477434 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:07.477103    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:11.477088398 +0000 UTC m=+239.757541170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : configmap "kube-rbac-proxy" not found
Sep 06 19:15:07.477434 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:07.477204    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:11.477190504 +0000 UTC m=+239.757643280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:07.515278 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:07.515249    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kvr6h_687926f7-d477-46b8-9142-c4a64f35d6e4/serve-healthcheck-canary/0.log"
Sep 06 19:15:07.679556 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:07.679506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:07.679776 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:07.679585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:07.679776 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:07.679675    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 19:15:07.679776 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:07.679695    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:15.679681825 +0000 UTC m=+243.960134598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : configmap references non-existent config key: service-ca.crt
Sep 06 19:15:07.679776 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:07.679739    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:15.679720715 +0000 UTC m=+243.960173488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : secret "router-metrics-certs-default" not found
Sep 06 19:15:07.767946 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:07.767856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:07.819162 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:07.819123    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:08.084769 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:08.084686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268" event={"ID":"ca149796-d1f4-4805-a50e-3fce6118c178","Type":"ContainerStarted","Data":"e78b92aac90d4ad264b6d2e71ae8c909f248e0fc929d043ba5de057970a36445"}
Sep 06 19:15:08.084769 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:08.084766    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268" event={"ID":"ca149796-d1f4-4805-a50e-3fce6118c178","Type":"ContainerStarted","Data":"59b6d956b1dff61ebbb178e3ff16dcd6a2f8350bba467e680b396dab4daed82c"}
Sep 06 19:15:08.100634 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:08.100586    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cz268" podStartSLOduration=1.824745538 podStartE2EDuration="3.100569978s" totalImagesPullingTime="1.27582444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:08.099668061 +0000 UTC m=+236.380120849" watchObservedRunningTime="2026-09-06 19:15:08.100569978 +0000 UTC m=+236.381022772"
Sep 06 19:15:08.174890 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:08.174859    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:08.226507 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:08.226473    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:08.611036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:08.610999    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:10.376168 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:10.376116    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:10.376168 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:10.376182    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:10.376598 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:10.376492    2581 scope.go:117] "RemoveContainer" containerID="db2cd57c310de88a24f94da8dd0c706c98ef69cfa18a08755a6bc43ce2127aee"
Sep 06 19:15:10.376651 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:10.376631    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-ttcnn_openshift-console-operator(3e3eadb1-92f7-4a9f-aa67-eab19946cbc0)\"" pod="openshift-console-operator/console-operator-85b454874b-ttcnn" podUID="3e3eadb1-92f7-4a9f-aa67-eab19946cbc0"
Sep 06 19:15:11.242549 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:11.242497    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:11.343288 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:11.343256    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:11.443488 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:11.443457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:11.509678 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:11.509595    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:11.509836 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:11.509719    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 19:15:11.509836 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:11.509730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:11.509836 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:11.509784    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:19.509765569 +0000 UTC m=+247.790218345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : configmap "kube-rbac-proxy" not found
Sep 06 19:15:11.509836 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:11.509831    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:11.509997 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:11.509865    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:19.509857643 +0000 UTC m=+247.790310415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:11.543877 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:11.543850    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:11.644604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:11.644574    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:11.934242 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:11.934196    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:12.932611 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:12.932570    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:13.326868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:13.326784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:13.602548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:13.602468    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:13.727432 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:13.727396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:14.131318 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:14.131282    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:14.666949 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:14.666912    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:14.767450 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:14.767415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:14.818346 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:14.818322    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:15.132188 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:15.132161    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:15.188950 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:15.188912    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:15.531109 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:15.531002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:15.748754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:15.748716    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:15.748984 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:15.748798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:15.748984 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:15.748944    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle podName:70c0a808-8fad-4a11-bcd1-4c79e8353373 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:31.748925869 +0000 UTC m=+260.029378658 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle") pod "router-default-576f88fb4d-zrv9b" (UID: "70c0a808-8fad-4a11-bcd1-4c79e8353373") : configmap references non-existent config key: service-ca.crt
Sep 06 19:15:15.751383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:15.751361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70c0a808-8fad-4a11-bcd1-4c79e8353373-metrics-certs\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:15.927446 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:15.927411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:16.126439 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:16.126406    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:16.731621 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:16.731588    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:16.948756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:16.948725    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:17.128650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:17.128570    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:17.527346 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:17.527305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:17.715241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:17.715203    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:17.770262 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:17.770233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:18.127108 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:18.127057    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:18.929414 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:18.929377    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:19.328578 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:19.328489    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:19.527447 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:19.527419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:19.578578 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:19.578538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:19.578578 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:19.578575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:19.578807 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:19.578716    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:19.578807 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:19.578799    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls podName:e621832d-7142-4720-a723-066c1cfb5adf nodeName:}" failed. No retries permitted until 2026-09-06 19:15:35.578777889 +0000 UTC m=+263.859230688 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g77gb" (UID: "e621832d-7142-4720-a723-066c1cfb5adf") : secret "insights-runtime-extractor-tls" not found
Sep 06 19:15:19.579189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:19.579171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e621832d-7142-4720-a723-066c1cfb5adf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:19.724541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:19.724506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:20.126019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:20.125985    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:20.725532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:20.725495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:21.326131 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:21.326103    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:21.528702 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:21.528671    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:21.726700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:21.726640    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:21.931079 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:21.931045    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:24.012686 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:24.012647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:15:24.015153 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:24.015133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88608533-3551-4eaf-8716-5757cd3108c0-metrics-certs\") pod \"network-metrics-daemon-xm5z7\" (UID: \"88608533-3551-4eaf-8716-5757cd3108c0\") " pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:15:24.263012 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:24.262917    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6p7v7\""
Sep 06 19:15:24.271022 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:24.271002    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm5z7"
Sep 06 19:15:24.541128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:24.541041    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xm5z7"]
Sep 06 19:15:24.544328 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:24.544297    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88608533_3551_4eaf_8716_5757cd3108c0.slice/crio-a209cc97c98a47b168dab6bbd059ad4515cc96028dcada324e49993bd170fe22 WatchSource:0}: Error finding container a209cc97c98a47b168dab6bbd059ad4515cc96028dcada324e49993bd170fe22: Status 404 returned error can't find the container with id a209cc97c98a47b168dab6bbd059ad4515cc96028dcada324e49993bd170fe22
Sep 06 19:15:25.144103 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:25.144052    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xm5z7" event={"ID":"88608533-3551-4eaf-8716-5757cd3108c0","Type":"ContainerStarted","Data":"a209cc97c98a47b168dab6bbd059ad4515cc96028dcada324e49993bd170fe22"}
Sep 06 19:15:25.260051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:25.260018    2581 scope.go:117] "RemoveContainer" containerID="db2cd57c310de88a24f94da8dd0c706c98ef69cfa18a08755a6bc43ce2127aee"
Sep 06 19:15:26.076080 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.076032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:26.149308 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.149269    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xm5z7" event={"ID":"88608533-3551-4eaf-8716-5757cd3108c0","Type":"ContainerStarted","Data":"2cca4afb5c5eb7fdbf1af3218b424d5d2cbfc92b2ab5bef895c611efbb06ce45"}
Sep 06 19:15:26.149773 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.149330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xm5z7" event={"ID":"88608533-3551-4eaf-8716-5757cd3108c0","Type":"ContainerStarted","Data":"0779eee2b24fb5bec4d683e95b5feac18cd91fc02f40a284cd2cea845c596b87"}
Sep 06 19:15:26.151469 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.151450    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:15:26.151560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.151506    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-ttcnn" event={"ID":"3e3eadb1-92f7-4a9f-aa67-eab19946cbc0","Type":"ContainerStarted","Data":"913f45c30eb488190eca84552f4a49d5809998a3d7272a208228ac006d5675cd"}
Sep 06 19:15:26.151765 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.151748    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:26.156491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.156470    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-ttcnn"
Sep 06 19:15:26.164157 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.164106    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xm5z7" podStartSLOduration=253.23217283 podStartE2EDuration="4m14.164092014s" totalImagesPullingTime="931.919184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:26.163129527 +0000 UTC m=+254.443582320" watchObservedRunningTime="2026-09-06 19:15:26.164092014 +0000 UTC m=+254.444544808"
Sep 06 19:15:26.177502 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.177444    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-ttcnn" podStartSLOduration=23.093693012 podStartE2EDuration="26.177428202s" totalImagesPullingTime="3.08373519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:26.177365736 +0000 UTC m=+254.457818554" watchObservedRunningTime="2026-09-06 19:15:26.177428202 +0000 UTC m=+254.457880998"
Sep 06 19:15:26.181769 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.181741    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:26.580407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.580367    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:26.760764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.760731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:26.813840 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:26.813804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:27.579206 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:27.579164    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:27.680563 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:27.680527    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:27.973375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:27.973342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:28.074104 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.074078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:28.174366 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.174335    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:28.460861 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.460827    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:28.757335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.757257    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-75xzf"]
Sep 06 19:15:28.760450 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.760432    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-75xzf"
Sep 06 19:15:28.762577 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.762555    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 06 19:15:28.762712 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.762673    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 06 19:15:28.762712 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.762673    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qwsb9\""
Sep 06 19:15:28.770988 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.770957    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-75xzf"]
Sep 06 19:15:28.816889 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.816856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:28.851024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.850986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25k2z\" (UniqueName: \"kubernetes.io/projected/23e1e106-0dcf-464c-97b0-34beec62031c-kube-api-access-25k2z\") pod \"downloads-85d98c998f-75xzf\" (UID: \"23e1e106-0dcf-464c-97b0-34beec62031c\") " pod="openshift-console/downloads-85d98c998f-75xzf"
Sep 06 19:15:28.883689 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.883655    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:28.951645 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.951609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25k2z\" (UniqueName: \"kubernetes.io/projected/23e1e106-0dcf-464c-97b0-34beec62031c-kube-api-access-25k2z\") pod \"downloads-85d98c998f-75xzf\" (UID: \"23e1e106-0dcf-464c-97b0-34beec62031c\") " pod="openshift-console/downloads-85d98c998f-75xzf"
Sep 06 19:15:28.959759 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.959731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25k2z\" (UniqueName: \"kubernetes.io/projected/23e1e106-0dcf-464c-97b0-34beec62031c-kube-api-access-25k2z\") pod \"downloads-85d98c998f-75xzf\" (UID: \"23e1e106-0dcf-464c-97b0-34beec62031c\") " pod="openshift-console/downloads-85d98c998f-75xzf"
Sep 06 19:15:28.983433 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:28.983397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:29.070102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:29.069984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-75xzf"
Sep 06 19:15:29.339761 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:29.339676    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-75xzf"]
Sep 06 19:15:29.342851 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:29.342819    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e1e106_0dcf_464c_97b0_34beec62031c.slice/crio-d4c7df8ef81926dc546cfddf528afc6be9cd46435efaf0c736bcee5e0a4c27c4 WatchSource:0}: Error finding container d4c7df8ef81926dc546cfddf528afc6be9cd46435efaf0c736bcee5e0a4c27c4: Status 404 returned error can't find the container with id d4c7df8ef81926dc546cfddf528afc6be9cd46435efaf0c736bcee5e0a4c27c4
Sep 06 19:15:29.576885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:29.576845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:29.678503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:29.678471    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:30.166398 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:30.166360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-75xzf" event={"ID":"23e1e106-0dcf-464c-97b0-34beec62031c","Type":"ContainerStarted","Data":"d4c7df8ef81926dc546cfddf528afc6be9cd46435efaf0c736bcee5e0a4c27c4"}
Sep 06 19:15:30.784600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:30.784566    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:30.884775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:30.884736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:31.776710 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:31.776664    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:31.777447 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:31.777417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c0a808-8fad-4a11-bcd1-4c79e8353373-service-ca-bundle\") pod \"router-default-576f88fb4d-zrv9b\" (UID: \"70c0a808-8fad-4a11-bcd1-4c79e8353373\") " pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:32.077495 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:32.077409    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8clg5\""
Sep 06 19:15:32.086237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:32.086206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:32.365859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:32.365694    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-576f88fb4d-zrv9b"]
Sep 06 19:15:32.368382 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:32.368355    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c0a808_8fad_4a11_bcd1_4c79e8353373.slice/crio-7230e6e3b8bf3aa8b78ffe9bdd547820ee7fcc86a5b1b659530af9a23b924dee WatchSource:0}: Error finding container 7230e6e3b8bf3aa8b78ffe9bdd547820ee7fcc86a5b1b659530af9a23b924dee: Status 404 returned error can't find the container with id 7230e6e3b8bf3aa8b78ffe9bdd547820ee7fcc86a5b1b659530af9a23b924dee
Sep 06 19:15:33.179375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:33.179227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-576f88fb4d-zrv9b" event={"ID":"70c0a808-8fad-4a11-bcd1-4c79e8353373","Type":"ContainerStarted","Data":"a516dc2acadd72279e6fabf7a2935489f5221f9117c4f0ef29409f40bc8be1eb"}
Sep 06 19:15:33.179375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:33.179349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-576f88fb4d-zrv9b" event={"ID":"70c0a808-8fad-4a11-bcd1-4c79e8353373","Type":"ContainerStarted","Data":"7230e6e3b8bf3aa8b78ffe9bdd547820ee7fcc86a5b1b659530af9a23b924dee"}
Sep 06 19:15:33.200046 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:33.199984    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-576f88fb4d-zrv9b" podStartSLOduration=34.19996685 podStartE2EDuration="34.19996685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:14:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:33.198788419 +0000 UTC m=+261.479241214" watchObservedRunningTime="2026-09-06 19:15:33.19996685 +0000 UTC m=+261.480419643"
Sep 06 19:15:34.086631 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.086595    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:34.090503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.090477    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:34.183493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.183447    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:34.184947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.184924    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-576f88fb4d-zrv9b"
Sep 06 19:15:34.552289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.552254    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:34.652228 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.652192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:34.695116 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.695056    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"]
Sep 06 19:15:34.698279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.698246    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"
Sep 06 19:15:34.700277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.700256    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 06 19:15:34.700388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.700331    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hvh9t\""
Sep 06 19:15:34.705139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.705075    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"]
Sep 06 19:15:34.717991 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.717961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:34.799789 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.799751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/244a06a3-600e-4488-9f00-fca365c23e84-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-ngvdx\" (UID: \"244a06a3-600e-4488-9f00-fca365c23e84\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"
Sep 06 19:15:34.900917 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:34.900872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/244a06a3-600e-4488-9f00-fca365c23e84-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-ngvdx\" (UID: \"244a06a3-600e-4488-9f00-fca365c23e84\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"
Sep 06 19:15:34.901098 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:34.901039    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 06 19:15:34.901163 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:34.901144    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244a06a3-600e-4488-9f00-fca365c23e84-tls-certificates podName:244a06a3-600e-4488-9f00-fca365c23e84 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:35.40112096 +0000 UTC m=+263.681573734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/244a06a3-600e-4488-9f00-fca365c23e84-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-ngvdx" (UID: "244a06a3-600e-4488-9f00-fca365c23e84") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 06 19:15:35.002042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.002002    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64cb479944-7zgtv"]
Sep 06 19:15:35.005381 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.005353    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.007642 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.007619    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 06 19:15:35.007779 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.007646    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nb98g\""
Sep 06 19:15:35.007841 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.007784    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 06 19:15:35.007925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.007903    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 06 19:15:35.008099 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.008055    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 06 19:15:35.008192 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.008158    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 06 19:15:35.014621 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.014599    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cb479944-7zgtv"]
Sep 06 19:15:35.102095 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.102047    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-oauth-config\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.102095 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.102097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-oauth-serving-cert\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.102353 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.102154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-serving-cert\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.102353 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.102170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5485x\" (UniqueName: \"kubernetes.io/projected/b037e792-6637-4021-9737-e2305570c10b-kube-api-access-5485x\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.102353 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.102196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-console-config\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.102353 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.102281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-service-ca\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.111134 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.111095    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:35.203412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.203321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-serving-cert\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.203412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.203363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5485x\" (UniqueName: \"kubernetes.io/projected/b037e792-6637-4021-9737-e2305570c10b-kube-api-access-5485x\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.203412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.203388    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-console-config\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.203936 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.203443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-service-ca\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.203936 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.203577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-oauth-config\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.203936 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.203622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-oauth-serving-cert\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.204488 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.204467    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-console-config\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.204652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.204546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-service-ca\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.204652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.204630    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-oauth-serving-cert\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.206281 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.206239    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-oauth-config\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.206441 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.206409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-serving-cert\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.212145 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.212124    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5485x\" (UniqueName: \"kubernetes.io/projected/b037e792-6637-4021-9737-e2305570c10b-kube-api-access-5485x\") pod \"console-64cb479944-7zgtv\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") " pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.316642 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.316604    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:35.405653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.405613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/244a06a3-600e-4488-9f00-fca365c23e84-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-ngvdx\" (UID: \"244a06a3-600e-4488-9f00-fca365c23e84\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"
Sep 06 19:15:35.408338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.408314    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/244a06a3-600e-4488-9f00-fca365c23e84-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-ngvdx\" (UID: \"244a06a3-600e-4488-9f00-fca365c23e84\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"
Sep 06 19:15:35.601328 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.601242    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cb479944-7zgtv"]
Sep 06 19:15:35.605040 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:35.605010    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb037e792_6637_4021_9737_e2305570c10b.slice/crio-d3bab8d489cac29298767202041c0f40b6bd63f0e52ade729dadac104f7cf553 WatchSource:0}: Error finding container d3bab8d489cac29298767202041c0f40b6bd63f0e52ade729dadac104f7cf553: Status 404 returned error can't find the container with id d3bab8d489cac29298767202041c0f40b6bd63f0e52ade729dadac104f7cf553
Sep 06 19:15:35.607518 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.607481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:35.610001 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.609983    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"
Sep 06 19:15:35.610181 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.610158    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e621832d-7142-4720-a723-066c1cfb5adf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g77gb\" (UID: \"e621832d-7142-4720-a723-066c1cfb5adf\") " pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:35.873026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.872940    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-grwwn\""
Sep 06 19:15:35.881631 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.881603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g77gb"
Sep 06 19:15:35.889845 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:35.889812    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"]
Sep 06 19:15:35.893217 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:35.893167    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244a06a3_600e_4488_9f00_fca365c23e84.slice/crio-9009f294bb155b67fb65aad63345da80e03b2c8c5fdf8733b240ccc19945921c WatchSource:0}: Error finding container 9009f294bb155b67fb65aad63345da80e03b2c8c5fdf8733b240ccc19945921c: Status 404 returned error can't find the container with id 9009f294bb155b67fb65aad63345da80e03b2c8c5fdf8733b240ccc19945921c
Sep 06 19:15:36.160623 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:36.160586    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g77gb"]
Sep 06 19:15:36.164707 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:36.164678    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode621832d_7142_4720_a723_066c1cfb5adf.slice/crio-4b5d234fdfb8f8e9abaf359f1c9564154f456184323001ed8e20ee98f5eb1ec7 WatchSource:0}: Error finding container 4b5d234fdfb8f8e9abaf359f1c9564154f456184323001ed8e20ee98f5eb1ec7: Status 404 returned error can't find the container with id 4b5d234fdfb8f8e9abaf359f1c9564154f456184323001ed8e20ee98f5eb1ec7
Sep 06 19:15:36.193208 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:36.193167    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g77gb" event={"ID":"e621832d-7142-4720-a723-066c1cfb5adf","Type":"ContainerStarted","Data":"4b5d234fdfb8f8e9abaf359f1c9564154f456184323001ed8e20ee98f5eb1ec7"}
Sep 06 19:15:36.195448 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:36.195412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx" event={"ID":"244a06a3-600e-4488-9f00-fca365c23e84","Type":"ContainerStarted","Data":"9009f294bb155b67fb65aad63345da80e03b2c8c5fdf8733b240ccc19945921c"}
Sep 06 19:15:36.197027 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:36.196972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cb479944-7zgtv" event={"ID":"b037e792-6637-4021-9737-e2305570c10b","Type":"ContainerStarted","Data":"d3bab8d489cac29298767202041c0f40b6bd63f0e52ade729dadac104f7cf553"}
Sep 06 19:15:37.203375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:37.203335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g77gb" event={"ID":"e621832d-7142-4720-a723-066c1cfb5adf","Type":"ContainerStarted","Data":"19970da0e78aaec5f593a5e07dd0e7b13710d94394e237397f0a7cd6d692316f"}
Sep 06 19:15:38.210173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:38.210128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g77gb" event={"ID":"e621832d-7142-4720-a723-066c1cfb5adf","Type":"ContainerStarted","Data":"1e1eae040d4b1adf4800989794280de597e58bf3e9fea73895be5b443e6ba536"}
Sep 06 19:15:38.212096 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:38.211984    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx" event={"ID":"244a06a3-600e-4488-9f00-fca365c23e84","Type":"ContainerStarted","Data":"4325ecb7606e97a6f9b831a68e65a20800718741020d1e732a22c44466e96679"}
Sep 06 19:15:38.212246 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:38.212195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"
Sep 06 19:15:38.218034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:38.218007    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx"
Sep 06 19:15:38.226324 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:38.226274    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-ngvdx" podStartSLOduration=2.907353776 podStartE2EDuration="4.226251094s" totalImagesPullingTime="1.318897318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:38.224596536 +0000 UTC m=+266.505049329" watchObservedRunningTime="2026-09-06 19:15:38.226251094 +0000 UTC m=+266.506703888"
Sep 06 19:15:38.717908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:38.717869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:38.785408 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:38.785370    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:38.838175 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:38.838145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:38.938826 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:38.938787    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:40.758986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:40.758950    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:41.519994 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:41.519959    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:41.620508 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:41.620474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:42.876008 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:42.875970    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:43.020398 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.020363    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:43.097733 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.097700    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:43.104890 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.102706    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l77bc"]
Sep 06 19:15:43.108244 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.108214    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"]
Sep 06 19:15:43.108384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.108325    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.110573 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.110528    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 06 19:15:43.110791 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.110770    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 06 19:15:43.110870 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.110834    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 06 19:15:43.110870 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.110847    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w8pjt\""
Sep 06 19:15:43.110870 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.110857    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 06 19:15:43.111002 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.110980    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 06 19:15:43.111397 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.111378    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 06 19:15:43.111675 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.111656    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.113633 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.113600    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 06 19:15:43.113785 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.113697    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 06 19:15:43.113785 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.113719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8jtxx\""
Sep 06 19:15:43.113934 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.113817    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 06 19:15:43.121596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.121572    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"]
Sep 06 19:15:43.227277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.227240    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:43.277328 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277282    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6903a9cf-ecd0-4d30-a595-4d32a89547c3-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.277328 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277334    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-sys\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.277557 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.277557 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.277557 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277517    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-wtmp\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.277557 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.277713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277576    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqt9j\" (UniqueName: \"kubernetes.io/projected/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-api-access-mqt9j\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.277713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-metrics-client-ca\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.277713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-tls\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.277713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277678    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-root\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.277713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.277895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2pcd\" (UniqueName: \"kubernetes.io/projected/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-kube-api-access-f2pcd\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.277895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-accelerators-collector-config\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.277895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-textfile\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.277895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.277861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6903a9cf-ecd0-4d30-a595-4d32a89547c3-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.379014 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.378978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-wtmp\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379014 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.379335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqt9j\" (UniqueName: \"kubernetes.io/projected/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-api-access-mqt9j\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.379335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-metrics-client-ca\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379208    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-wtmp\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379216    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-tls\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-root\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379335 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:43.379319    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2pcd\" (UniqueName: \"kubernetes.io/projected/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-kube-api-access-f2pcd\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-accelerators-collector-config\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-root\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:43.379391    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-tls podName:4060c3dc-9d38-4f92-bde1-d20bd499c9b7 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:43.879372008 +0000 UTC m=+272.159824783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-tls") pod "node-exporter-l77bc" (UID: "4060c3dc-9d38-4f92-bde1-d20bd499c9b7") : secret "node-exporter-tls" not found
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379543    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-textfile\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6903a9cf-ecd0-4d30-a595-4d32a89547c3-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379610    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6903a9cf-ecd0-4d30-a595-4d32a89547c3-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-sys\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379660    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.379738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.380190 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379778    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-sys\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.380190 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379808    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-metrics-client-ca\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.380190 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:43.379831    2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 06 19:15:43.380190 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:43.379865    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-tls podName:6903a9cf-ecd0-4d30-a595-4d32a89547c3 nodeName:}" failed. No retries permitted until 2026-09-06 19:15:43.879854096 +0000 UTC m=+272.160306867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-2s4kx" (UID: "6903a9cf-ecd0-4d30-a595-4d32a89547c3") : secret "kube-state-metrics-tls" not found
Sep 06 19:15:43.380190 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.379874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.380190 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.380031    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-accelerators-collector-config\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.380484 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.380197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-textfile\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.380484 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.380306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6903a9cf-ecd0-4d30-a595-4d32a89547c3-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.380484 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.380368    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6903a9cf-ecd0-4d30-a595-4d32a89547c3-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.382245 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.382201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.382716 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.382688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.387522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.387499    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqt9j\" (UniqueName: \"kubernetes.io/projected/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-api-access-mqt9j\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.387867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.387843    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2pcd\" (UniqueName: \"kubernetes.io/projected/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-kube-api-access-f2pcd\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.454229 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.454191    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:43.554208 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.554126    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:43.607658 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.607602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:43.884382 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.884344    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.884839 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.884446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-tls\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:43.887300 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.887277    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6903a9cf-ecd0-4d30-a595-4d32a89547c3-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2s4kx\" (UID: \"6903a9cf-ecd0-4d30-a595-4d32a89547c3\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:43.887300 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:43.887297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4060c3dc-9d38-4f92-bde1-d20bd499c9b7-node-exporter-tls\") pod \"node-exporter-l77bc\" (UID: \"4060c3dc-9d38-4f92-bde1-d20bd499c9b7\") " pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:44.021293 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:44.021258    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l77bc"
Sep 06 19:15:44.028599 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:44.028567    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"
Sep 06 19:15:44.064350 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:44.064307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:44.147665 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:44.147582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:44.262280 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:44.262226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:45.075447 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.075411    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"]
Sep 06 19:15:45.079338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.079304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.081522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.081486    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 06 19:15:45.081657 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.081541    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 06 19:15:45.081657 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.081556    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 06 19:15:45.081657 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.081571    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 06 19:15:45.081834 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.081670    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 06 19:15:45.081918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.081879    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-58zsk\""
Sep 06 19:15:45.081980 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.081913    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cpnuf2nib2f45\""
Sep 06 19:15:45.089827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.089805    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"]
Sep 06 19:15:45.091638 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.091611    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.091759 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.091649    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.091759 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.091670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.091865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.091765    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.091865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.091798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfprk\" (UniqueName: \"kubernetes.io/projected/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-kube-api-access-xfprk\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.091865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.091831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-tls\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.092020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.091874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-metrics-client-ca\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.092020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.091945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-grpc-tls\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.185283 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.185245    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fdb697ff4-7776p"]
Sep 06 19:15:45.188710 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.188687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.192333 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.192465 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.192465 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.192465 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-service-ca\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.192673 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.192791 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfprk\" (UniqueName: \"kubernetes.io/projected/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-kube-api-access-xfprk\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.192791 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-tls\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.192791 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-oauth-config\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.192791 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-trusted-ca-bundle\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.193020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-oauth-serving-cert\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.193020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-console-config\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.193020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf9qb\" (UniqueName: \"kubernetes.io/projected/98520c34-904c-472d-a746-c57d372ebd89-kube-api-access-pf9qb\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.193020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-metrics-client-ca\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.193020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-grpc-tls\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.193020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.192992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-serving-cert\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.194535 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.194443    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-metrics-client-ca\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.195844 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.195801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.195945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.195928    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.196491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.196470    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 06 19:15:45.196592 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.196513    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.197258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.197238    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.198134 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.198042    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-grpc-tls\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.198934 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.198911    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-secret-thanos-querier-tls\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.200211 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.200164    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fdb697ff4-7776p"]
Sep 06 19:15:45.202801 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.202776    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfprk\" (UniqueName: \"kubernetes.io/projected/30a91bfe-84fd-4a0c-8bf1-07a7655a303d-kube-api-access-xfprk\") pod \"thanos-querier-648c8d56f9-4vnhn\" (UID: \"30a91bfe-84fd-4a0c-8bf1-07a7655a303d\") " pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.293520 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.293480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-serving-cert\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.293713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.293540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-service-ca\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.293713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.293604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-oauth-config\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.293713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.293634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-trusted-ca-bundle\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.293713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.293679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-oauth-serving-cert\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.293929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.293721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-console-config\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.293929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.293752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf9qb\" (UniqueName: \"kubernetes.io/projected/98520c34-904c-472d-a746-c57d372ebd89-kube-api-access-pf9qb\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.294548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.294519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-service-ca\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.294663 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.294642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-oauth-serving-cert\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.294742 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.294723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-console-config\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.295351 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.295327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-trusted-ca-bundle\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.296886 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.296833    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-serving-cert\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.296886 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.296863    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-oauth-config\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.301573 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.301528    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf9qb\" (UniqueName: \"kubernetes.io/projected/98520c34-904c-472d-a746-c57d372ebd89-kube-api-access-pf9qb\") pod \"console-5fdb697ff4-7776p\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") " pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:45.391218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.391180    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:45.520086 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:45.520030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:46.020279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:46.020239    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:46.120538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:46.120496    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:46.194240 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:46.194205    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:46.338493 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:46.338443    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4060c3dc_9d38_4f92_bde1_d20bd499c9b7.slice/crio-9ecf426cee4c62d2b79648fa7ccf00fb92dde321143aa6755ed21870f444446e WatchSource:0}: Error finding container 9ecf426cee4c62d2b79648fa7ccf00fb92dde321143aa6755ed21870f444446e: Status 404 returned error can't find the container with id 9ecf426cee4c62d2b79648fa7ccf00fb92dde321143aa6755ed21870f444446e
Sep 06 19:15:46.587535 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:46.587407    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:46.612092 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:46.611454    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx"]
Sep 06 19:15:46.615767 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:46.615732    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6903a9cf_ecd0_4d30_a595_4d32a89547c3.slice/crio-64416c84f44a7974dbb436e3a38faf0c5c00a8d81baecf08a9ab68d29b6072fe WatchSource:0}: Error finding container 64416c84f44a7974dbb436e3a38faf0c5c00a8d81baecf08a9ab68d29b6072fe: Status 404 returned error can't find the container with id 64416c84f44a7974dbb436e3a38faf0c5c00a8d81baecf08a9ab68d29b6072fe
Sep 06 19:15:46.625154 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:46.625001    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fdb697ff4-7776p"]
Sep 06 19:15:46.627811 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:46.627782    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"]
Sep 06 19:15:46.628275 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:46.628249    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98520c34_904c_472d_a746_c57d372ebd89.slice/crio-9b9bbabda1ea1103414bc1b6e796be739421124c22eb8404c24f9a182b3f2366 WatchSource:0}: Error finding container 9b9bbabda1ea1103414bc1b6e796be739421124c22eb8404c24f9a182b3f2366: Status 404 returned error can't find the container with id 9b9bbabda1ea1103414bc1b6e796be739421124c22eb8404c24f9a182b3f2366
Sep 06 19:15:46.633421 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:46.633396    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a91bfe_84fd_4a0c_8bf1_07a7655a303d.slice/crio-69585808bfdb990fc6af0171c267c3457e716bbe50937b2bab9d83bb95c382e2 WatchSource:0}: Error finding container 69585808bfdb990fc6af0171c267c3457e716bbe50937b2bab9d83bb95c382e2: Status 404 returned error can't find the container with id 69585808bfdb990fc6af0171c267c3457e716bbe50937b2bab9d83bb95c382e2
Sep 06 19:15:47.253102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.252734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cb479944-7zgtv" event={"ID":"b037e792-6637-4021-9737-e2305570c10b","Type":"ContainerStarted","Data":"f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287"}
Sep 06 19:15:47.258083 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.257430    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb697ff4-7776p" event={"ID":"98520c34-904c-472d-a746-c57d372ebd89","Type":"ContainerStarted","Data":"da308ad6780b89ea6b9bf7a6577d34f4593f27c72690755bc03c89b629753b36"}
Sep 06 19:15:47.258083 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.257489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb697ff4-7776p" event={"ID":"98520c34-904c-472d-a746-c57d372ebd89","Type":"ContainerStarted","Data":"9b9bbabda1ea1103414bc1b6e796be739421124c22eb8404c24f9a182b3f2366"}
Sep 06 19:15:47.260634 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.260585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn" event={"ID":"30a91bfe-84fd-4a0c-8bf1-07a7655a303d","Type":"ContainerStarted","Data":"69585808bfdb990fc6af0171c267c3457e716bbe50937b2bab9d83bb95c382e2"}
Sep 06 19:15:47.265156 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.264589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-75xzf" event={"ID":"23e1e106-0dcf-464c-97b0-34beec62031c","Type":"ContainerStarted","Data":"2db6d621006c645fac18170bacf65b06f36400d008a7971d90113b1899344422"}
Sep 06 19:15:47.265156 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.265055    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-75xzf"
Sep 06 19:15:47.267910 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.267878    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l77bc" event={"ID":"4060c3dc-9d38-4f92-bde1-d20bd499c9b7","Type":"ContainerStarted","Data":"9ecf426cee4c62d2b79648fa7ccf00fb92dde321143aa6755ed21870f444446e"}
Sep 06 19:15:47.269646 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.269583    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx" event={"ID":"6903a9cf-ecd0-4d30-a595-4d32a89547c3","Type":"ContainerStarted","Data":"64416c84f44a7974dbb436e3a38faf0c5c00a8d81baecf08a9ab68d29b6072fe"}
Sep 06 19:15:47.272347 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.272293    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64cb479944-7zgtv" podStartSLOduration=2.539858923 podStartE2EDuration="13.272275326s" totalImagesPullingTime="10.732416403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:47.271377223 +0000 UTC m=+275.551830019" watchObservedRunningTime="2026-09-06 19:15:47.272275326 +0000 UTC m=+275.552728121"
Sep 06 19:15:47.275464 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.275381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g77gb" event={"ID":"e621832d-7142-4720-a723-066c1cfb5adf","Type":"ContainerStarted","Data":"aa02d49afa61d9f2813fd25790789782fa6d0b31fac9054898343bbacd90dcc8"}
Sep 06 19:15:47.278155 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.278131    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-75xzf"
Sep 06 19:15:47.285631 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.285531    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-75xzf" podStartSLOduration=2.29067349 podStartE2EDuration="19.285503116s" totalImagesPullingTime="16.994829626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:47.285164902 +0000 UTC m=+275.565617696" watchObservedRunningTime="2026-09-06 19:15:47.285503116 +0000 UTC m=+275.565955912"
Sep 06 19:15:47.310322 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.308440    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fdb697ff4-7776p" podStartSLOduration=2.30841908 podStartE2EDuration="2.30841908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:47.307171637 +0000 UTC m=+275.587624456" watchObservedRunningTime="2026-09-06 19:15:47.30841908 +0000 UTC m=+275.588871874"
Sep 06 19:15:47.327030 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.325105    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g77gb" podStartSLOduration=34.248822163 podStartE2EDuration="44.325083785s" totalImagesPullingTime="10.076261622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:15:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:47.324056885 +0000 UTC m=+275.604509692" watchObservedRunningTime="2026-09-06 19:15:47.325083785 +0000 UTC m=+275.605536581"
Sep 06 19:15:47.495800 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.495770    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6d998fc79-f5pnk"]
Sep 06 19:15:47.503814 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.502444    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.505282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.505258    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 06 19:15:47.505392 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.505328    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 06 19:15:47.505392 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.505339    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 06 19:15:47.505512 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.505258    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-d7j7g\""
Sep 06 19:15:47.510963 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.509054    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d61654b3c0qn5\""
Sep 06 19:15:47.510963 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.509740    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 06 19:15:47.511694 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.511551    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d998fc79-f5pnk"]
Sep 06 19:15:47.612541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.612481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c2241bb1-852d-40cb-ad75-74953f81179c-secret-metrics-server-client-certs\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.612541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.612540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c2241bb1-852d-40cb-ad75-74953f81179c-secret-metrics-server-tls\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.612765 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.612585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2241bb1-852d-40cb-ad75-74953f81179c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.612765 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.612650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2241bb1-852d-40cb-ad75-74953f81179c-client-ca-bundle\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.612765 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.612686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c2241bb1-852d-40cb-ad75-74953f81179c-metrics-server-audit-profiles\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.612765 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.612723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5zdl\" (UniqueName: \"kubernetes.io/projected/c2241bb1-852d-40cb-ad75-74953f81179c-kube-api-access-p5zdl\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.612974 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.612774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c2241bb1-852d-40cb-ad75-74953f81179c-audit-log\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.713979 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.713935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c2241bb1-852d-40cb-ad75-74953f81179c-secret-metrics-server-client-certs\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.714204 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.713999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c2241bb1-852d-40cb-ad75-74953f81179c-secret-metrics-server-tls\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.714204 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.714038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2241bb1-852d-40cb-ad75-74953f81179c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.714204 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.714098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2241bb1-852d-40cb-ad75-74953f81179c-client-ca-bundle\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.714204 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.714131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c2241bb1-852d-40cb-ad75-74953f81179c-metrics-server-audit-profiles\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.714204 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.714177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5zdl\" (UniqueName: \"kubernetes.io/projected/c2241bb1-852d-40cb-ad75-74953f81179c-kube-api-access-p5zdl\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.714481 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.714230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c2241bb1-852d-40cb-ad75-74953f81179c-audit-log\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.714752 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.714693    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c2241bb1-852d-40cb-ad75-74953f81179c-audit-log\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.715781 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.715742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c2241bb1-852d-40cb-ad75-74953f81179c-metrics-server-audit-profiles\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.716524 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.716477    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2241bb1-852d-40cb-ad75-74953f81179c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.720979 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.720950    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c2241bb1-852d-40cb-ad75-74953f81179c-secret-metrics-server-tls\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.725385 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.725339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2241bb1-852d-40cb-ad75-74953f81179c-client-ca-bundle\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.726163 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.726121    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5zdl\" (UniqueName: \"kubernetes.io/projected/c2241bb1-852d-40cb-ad75-74953f81179c-kube-api-access-p5zdl\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.727733 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.727674    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c2241bb1-852d-40cb-ad75-74953f81179c-secret-metrics-server-client-certs\") pod \"metrics-server-6d998fc79-f5pnk\" (UID: \"c2241bb1-852d-40cb-ad75-74953f81179c\") " pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.841654 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.841574    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:15:47.889287 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.889202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:47.895025 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.894056    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"]
Sep 06 19:15:47.915473 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.915412    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"]
Sep 06 19:15:47.915641 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.915545    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:47.918087 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.918021    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 06 19:15:47.918211 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.918089    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-s6575\""
Sep 06 19:15:47.945008 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:47.944966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:48.018114 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.017801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28288a9d-eb7e-4806-abf2-41a17b9a21ae-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-4wrd5\" (UID: \"28288a9d-eb7e-4806-abf2-41a17b9a21ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:48.018114 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.017851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2zbc\" (UniqueName: \"kubernetes.io/projected/28288a9d-eb7e-4806-abf2-41a17b9a21ae-kube-api-access-p2zbc\") pod \"monitoring-plugin-85bbbbbf4-4wrd5\" (UID: \"28288a9d-eb7e-4806-abf2-41a17b9a21ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:48.120255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.119213    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28288a9d-eb7e-4806-abf2-41a17b9a21ae-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-4wrd5\" (UID: \"28288a9d-eb7e-4806-abf2-41a17b9a21ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:48.120255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.119268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2zbc\" (UniqueName: \"kubernetes.io/projected/28288a9d-eb7e-4806-abf2-41a17b9a21ae-kube-api-access-p2zbc\") pod \"monitoring-plugin-85bbbbbf4-4wrd5\" (UID: \"28288a9d-eb7e-4806-abf2-41a17b9a21ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:48.120255 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:48.119905    2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 06 19:15:48.120255 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:48.119963    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28288a9d-eb7e-4806-abf2-41a17b9a21ae-monitoring-plugin-cert podName:28288a9d-eb7e-4806-abf2-41a17b9a21ae nodeName:}" failed. No retries permitted until 2026-09-06 19:15:48.619948301 +0000 UTC m=+276.900401073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/28288a9d-eb7e-4806-abf2-41a17b9a21ae-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-4wrd5" (UID: "28288a9d-eb7e-4806-abf2-41a17b9a21ae") : secret "monitoring-plugin-cert" not found
Sep 06 19:15:48.128206 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.128173    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2zbc\" (UniqueName: \"kubernetes.io/projected/28288a9d-eb7e-4806-abf2-41a17b9a21ae-kube-api-access-p2zbc\") pod \"monitoring-plugin-85bbbbbf4-4wrd5\" (UID: \"28288a9d-eb7e-4806-abf2-41a17b9a21ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:48.207235 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.207160    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:48.284332 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.284287    2581 generic.go:356] "Generic (PLEG): container finished" podID="4060c3dc-9d38-4f92-bde1-d20bd499c9b7" containerID="9933ba041238a5edeee8b6270d678c5a56fc3045b1492d00b17b90592219c926" exitCode=0
Sep 06 19:15:48.286580 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.286013    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l77bc" event={"ID":"4060c3dc-9d38-4f92-bde1-d20bd499c9b7","Type":"ContainerDied","Data":"9933ba041238a5edeee8b6270d678c5a56fc3045b1492d00b17b90592219c926"}
Sep 06 19:15:48.626872 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.626833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28288a9d-eb7e-4806-abf2-41a17b9a21ae-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-4wrd5\" (UID: \"28288a9d-eb7e-4806-abf2-41a17b9a21ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:48.630236 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.630168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28288a9d-eb7e-4806-abf2-41a17b9a21ae-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-4wrd5\" (UID: \"28288a9d-eb7e-4806-abf2-41a17b9a21ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:48.634932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.634838    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d998fc79-f5pnk"]
Sep 06 19:15:48.830602 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:48.830557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:49.269197 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.269155    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 19:15:49.271979 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.271951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:49.279596 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:49.279562    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2241bb1_852d_40cb_ad75_74953f81179c.slice/crio-7dcc83731adc9b5ddb9f1497098c9fc0c228fcb111a577ef942eb88bdbd67d8f WatchSource:0}: Error finding container 7dcc83731adc9b5ddb9f1497098c9fc0c228fcb111a577ef942eb88bdbd67d8f: Status 404 returned error can't find the container with id 7dcc83731adc9b5ddb9f1497098c9fc0c228fcb111a577ef942eb88bdbd67d8f
Sep 06 19:15:49.294042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.294006    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 19:15:49.294502 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.294096    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l77bc" event={"ID":"4060c3dc-9d38-4f92-bde1-d20bd499c9b7","Type":"ContainerStarted","Data":"487c73dc55263bbe980eaa8b8d2be2514b19e5cd783a599126f87043467043f4"}
Sep 06 19:15:49.294502 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.294129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk" event={"ID":"c2241bb1-852d-40cb-ad75-74953f81179c","Type":"ContainerStarted","Data":"7dcc83731adc9b5ddb9f1497098c9fc0c228fcb111a577ef942eb88bdbd67d8f"}
Sep 06 19:15:49.294502 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.294239    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.296816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.296509    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 06 19:15:49.296816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.296574    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 06 19:15:49.296816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.296596    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m2xbg\""
Sep 06 19:15:49.297295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.297270    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 06 19:15:49.297519 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.297499    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 06 19:15:49.297624 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.297598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 06 19:15:49.297689 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.297644    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 06 19:15:49.297774 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.297750    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 06 19:15:49.298039 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.298019    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 06 19:15:49.298818 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.298187    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 06 19:15:49.298818 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.297763    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 06 19:15:49.298818 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.298309    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 06 19:15:49.298818 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.298340    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 06 19:15:49.298818 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.298378    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 06 19:15:49.299129 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.298838    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4ogclut4d2o01\""
Sep 06 19:15:49.300051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.300034    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 06 19:15:49.333195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-config\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333195    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-web-config\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333284    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333429    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333476    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333512    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgrsq\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-kube-api-access-hgrsq\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333602    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333636    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333719    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.333813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.334230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.334230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.333839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-config-out\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.435286 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgrsq\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-kube-api-access-hgrsq\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435467    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-config-out\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-config\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435692    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-web-config\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.437348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.438265 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.438265 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.435914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.438265 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.436362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.438265 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.436501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.438265 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.436905    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.438525 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.438350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.440018 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.438599    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.440018 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.439282    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.440018 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.439458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.440018 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.439828    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.440018 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.439997    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.445417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.441340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-web-config\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.445417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.443002    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.445417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.443430    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.445417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.443564    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.445417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.443608    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.445417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.443724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-config\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.445417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.444666    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-config-out\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.445417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.445138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.446722 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.446384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.448444 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.448419    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.449722 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.449697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgrsq\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-kube-api-access-hgrsq\") pod \"prometheus-k8s-0\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.607845 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.607554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:49.871665 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.868905    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"]
Sep 06 19:15:49.956432 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:49.956337    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 19:15:50.300918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.300758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5" event={"ID":"28288a9d-eb7e-4806-abf2-41a17b9a21ae","Type":"ContainerStarted","Data":"473c7623a2fc962c58f711c70f607c772e75ac2c09c773af06d40a7a9d0bca95"}
Sep 06 19:15:50.304186 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.304153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l77bc" event={"ID":"4060c3dc-9d38-4f92-bde1-d20bd499c9b7","Type":"ContainerStarted","Data":"c2ab3051f39accd208e245d43a6ffde7659c7342a276d0c14d08d48580473214"}
Sep 06 19:15:50.307829 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.307800    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx" event={"ID":"6903a9cf-ecd0-4d30-a595-4d32a89547c3","Type":"ContainerStarted","Data":"4b5c9c3f5c750141322deef974d3fb10ce6f89ee46bebdc4961037efe4baab81"}
Sep 06 19:15:50.307989 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.307853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx" event={"ID":"6903a9cf-ecd0-4d30-a595-4d32a89547c3","Type":"ContainerStarted","Data":"d8dfeec19197ad7f7165dc0b9fc9adbc0bcfd0b243bff853253db27c166a947b"}
Sep 06 19:15:50.307989 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.307882    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx" event={"ID":"6903a9cf-ecd0-4d30-a595-4d32a89547c3","Type":"ContainerStarted","Data":"3a4d3656b109781c6869303d2f220e32e53509b64462207b9b62456a6cc7921a"}
Sep 06 19:15:50.310226 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.310146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerStarted","Data":"17eede25ea368687df6a119937db526ecfb85f3c6f04ea0886804adc7200244d"}
Sep 06 19:15:50.313321 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.313285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn" event={"ID":"30a91bfe-84fd-4a0c-8bf1-07a7655a303d","Type":"ContainerStarted","Data":"8a5a865b23e2f94f325e79793c45c96ab77a190d8c8cdd8e1ee68a9ab43a0aaf"}
Sep 06 19:15:50.313554 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.313358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn" event={"ID":"30a91bfe-84fd-4a0c-8bf1-07a7655a303d","Type":"ContainerStarted","Data":"b60f62e120a6be6c66de77306c6bcbad1f11906d9eed1e2c8dafdcfad67dc5f0"}
Sep 06 19:15:50.313554 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.313401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn" event={"ID":"30a91bfe-84fd-4a0c-8bf1-07a7655a303d","Type":"ContainerStarted","Data":"97ed0a5588a415cf5c8ed699460cbb1d7d3fa197f47ce8dfddaa55d328cde7b2"}
Sep 06 19:15:50.323787 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.322352    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l77bc" podStartSLOduration=6.1045313 podStartE2EDuration="7.322331704s" totalImagesPullingTime="1.125200207s" totalInitContainerRuntime="92.600197ms" isStatefulPod=true podCreationTimestamp="2026-09-06 19:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:50.321410772 +0000 UTC m=+278.601863591" watchObservedRunningTime="2026-09-06 19:15:50.322331704 +0000 UTC m=+278.602784501"
Sep 06 19:15:50.338393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:50.338342    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2s4kx" podStartSLOduration=4.366495276 podStartE2EDuration="7.338322161s" totalImagesPullingTime="2.971826885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:50.337380127 +0000 UTC m=+278.617832916" watchObservedRunningTime="2026-09-06 19:15:50.338322161 +0000 UTC m=+278.618774956"
Sep 06 19:15:52.765652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:52.765614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:52.867457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:52.867414    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:52.917192 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:52.917156    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:52.999381 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:52.999340    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64cb479944-7zgtv"]
Sep 06 19:15:53.870581 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:15:53.870541    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7xc4f" podUID="cce4e1bc-30ea-4560-813d-c7c07a1f368e"
Sep 06 19:15:54.339035 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.339005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5" event={"ID":"28288a9d-eb7e-4806-abf2-41a17b9a21ae","Type":"ContainerStarted","Data":"7b3b033bd1328e942f5ec0d248818d3bd451b5ce053f0a18c6dfe6918686fa9a"}
Sep 06 19:15:54.339283 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.339205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:54.341625 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.341594    2581 generic.go:356] "Generic (PLEG): container finished" podID="186bcb5f-0915-4769-acc3-34a69b334423" containerID="561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b" exitCode=0
Sep 06 19:15:54.341761 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.341687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerDied","Data":"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"}
Sep 06 19:15:54.345887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.345858    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk" event={"ID":"c2241bb1-852d-40cb-ad75-74953f81179c","Type":"ContainerStarted","Data":"5cf4d9f0be365973df90e4136def30ff63f64b5edb67cfa589c77b1bc4dc6b85"}
Sep 06 19:15:54.346554 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.346532    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5"
Sep 06 19:15:54.350387 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.350359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn" event={"ID":"30a91bfe-84fd-4a0c-8bf1-07a7655a303d","Type":"ContainerStarted","Data":"cf2cadbfd1a959b2365a4335335db488bb200ffcd17733be4cee59d6ed69044e"}
Sep 06 19:15:54.350495 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.350431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn" event={"ID":"30a91bfe-84fd-4a0c-8bf1-07a7655a303d","Type":"ContainerStarted","Data":"64f162566048f75e349a01ca26d4d531f27e135e932da56f1074890a2bd65c2c"}
Sep 06 19:15:54.350495 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.350374    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:15:54.350495 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.350471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn" event={"ID":"30a91bfe-84fd-4a0c-8bf1-07a7655a303d","Type":"ContainerStarted","Data":"f42a0332c910c75ec7c2012dfaf93a19d7fa7c4c5420d3736ed7a5e6d3391281"}
Sep 06 19:15:54.350980 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.350960    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:54.355497 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.355423    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4wrd5" podStartSLOduration=3.648578792 podStartE2EDuration="7.355405524s" totalImagesPullingTime="3.706826732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:54.353539559 +0000 UTC m=+282.633992355" watchObservedRunningTime="2026-09-06 19:15:54.355405524 +0000 UTC m=+282.635858314"
Sep 06 19:15:54.357792 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.357767    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn"
Sep 06 19:15:54.373754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.373697    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-648c8d56f9-4vnhn" podStartSLOduration=2.9220422790000002 podStartE2EDuration="9.373675896s" totalImagesPullingTime="6.451633617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:54.371510148 +0000 UTC m=+282.651962942" watchObservedRunningTime="2026-09-06 19:15:54.373675896 +0000 UTC m=+282.654128691"
Sep 06 19:15:54.394086 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:54.390179    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk" podStartSLOduration=3.392709968 podStartE2EDuration="7.390150785s" totalImagesPullingTime="3.997440817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:54.386206494 +0000 UTC m=+282.666659357" watchObservedRunningTime="2026-09-06 19:15:54.390150785 +0000 UTC m=+282.670603571"
Sep 06 19:15:55.317726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:55.317668    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:15:55.520568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:55.520491    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:55.520568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:55.520571    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:15:55.522186 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:55.522141    2581 patch_prober.go:28] interesting pod/console-5fdb697ff4-7776p container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" start-of-body=
Sep 06 19:15:55.522329 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:55.522199    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5fdb697ff4-7776p" podUID="98520c34-904c-472d-a746-c57d372ebd89" containerName="console" probeResult="failure" output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused"
Sep 06 19:15:56.922052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:56.922012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:15:56.924871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:56.924840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cce4e1bc-30ea-4560-813d-c7c07a1f368e-metrics-tls\") pod \"dns-default-7xc4f\" (UID: \"cce4e1bc-30ea-4560-813d-c7c07a1f368e\") " pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:15:57.053739 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:57.053707    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nxfsd\""
Sep 06 19:15:57.061369 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:57.061330    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:15:57.397216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:57.397170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:57.497082 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:57.497035    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:15:57.824770 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:57.824746    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xc4f"]
Sep 06 19:15:57.924970 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:15:57.924940    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce4e1bc_30ea_4560_813d_c7c07a1f368e.slice/crio-eb7d1369dea01c5f54fb9e5d32ad399ba6b6e6a0a99140b6cfeea03904104a8e WatchSource:0}: Error finding container eb7d1369dea01c5f54fb9e5d32ad399ba6b6e6a0a99140b6cfeea03904104a8e: Status 404 returned error can't find the container with id eb7d1369dea01c5f54fb9e5d32ad399ba6b6e6a0a99140b6cfeea03904104a8e
Sep 06 19:15:58.374652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:58.374608    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerStarted","Data":"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"}
Sep 06 19:15:58.374821 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:58.374705    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerStarted","Data":"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"}
Sep 06 19:15:58.374821 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:58.374746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerStarted","Data":"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"}
Sep 06 19:15:58.376108 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:58.376083    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xc4f" event={"ID":"cce4e1bc-30ea-4560-813d-c7c07a1f368e","Type":"ContainerStarted","Data":"eb7d1369dea01c5f54fb9e5d32ad399ba6b6e6a0a99140b6cfeea03904104a8e"}
Sep 06 19:15:59.385166 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:59.385124    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerStarted","Data":"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"}
Sep 06 19:15:59.385652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:59.385213    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerStarted","Data":"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"}
Sep 06 19:15:59.385652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:59.385250    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerStarted","Data":"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"}
Sep 06 19:15:59.413958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:59.413889    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.150248761 podStartE2EDuration="10.413864177s" totalImagesPullingTime="7.224512512s" totalInitContainerRuntime="39.102904ms" isStatefulPod=false podCreationTimestamp="2026-09-06 19:15:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:15:59.411471077 +0000 UTC m=+287.691923922" watchObservedRunningTime="2026-09-06 19:15:59.413864177 +0000 UTC m=+287.694316973"
Sep 06 19:15:59.608283 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:59.608240    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:15:59.765644 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:15:59.765567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:00.390923 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:00.390887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xc4f" event={"ID":"cce4e1bc-30ea-4560-813d-c7c07a1f368e","Type":"ContainerStarted","Data":"5f55eb416e6b4443501d71df6737f3d70f132f35577a6f982abea03b785d9f09"}
Sep 06 19:16:00.391416 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:00.390949    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xc4f" event={"ID":"cce4e1bc-30ea-4560-813d-c7c07a1f368e","Type":"ContainerStarted","Data":"8f6f339f0b67c917ae1a70f21ccbc8dd68867444008c09b79532f125d6306898"}
Sep 06 19:16:00.391477 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:00.391418    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:16:00.407244 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:00.407191    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7xc4f" podStartSLOduration=251.317920582 podStartE2EDuration="4m13.407173959s" totalImagesPullingTime="2.089253377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:16:00.405325105 +0000 UTC m=+288.685777899" watchObservedRunningTime="2026-09-06 19:16:00.407173959 +0000 UTC m=+288.687626756"
Sep 06 19:16:01.406223 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:01.406187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:01.541126 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:01.541090    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:01.641930 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:01.641895    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:04.106549 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:04.106514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:04.158411 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:04.158381    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:04.285466 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:04.285426    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fdb697ff4-7776p"]
Sep 06 19:16:04.381648 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:04.381555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:04.481675 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:04.481646    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:04.583257 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:04.583226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:05.066025 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:05.065996    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:05.165919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:05.165873    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:07.841959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:07.841921    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:16:07.842370 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:07.841986    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:16:10.274474 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:10.274440    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:10.331938 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:10.331901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:11.353085 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:11.353034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:11.401980 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:11.401955    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7xc4f"
Sep 06 19:16:11.420519 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:11.420488    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:12.173684 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:12.173654    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:16:12.173928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:12.173663    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:16:12.179492 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:12.179467    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:16:12.179648 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:12.179629    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:16:12.188195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:12.188176    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 06 19:16:14.582705 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:14.582665    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:14.634723 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:14.634686    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:15.455732 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:15.455699    2581 generic.go:356] "Generic (PLEG): container finished" podID="e97e47da-6bc6-48f8-bf89-18da9c037980" containerID="e6f70f18cf94611867449a0d36c5d0a9d58c642c0c3c3cbec810e78e12a37bdb" exitCode=0
Sep 06 19:16:15.455897 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:15.455775    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm" event={"ID":"e97e47da-6bc6-48f8-bf89-18da9c037980","Type":"ContainerDied","Data":"e6f70f18cf94611867449a0d36c5d0a9d58c642c0c3c3cbec810e78e12a37bdb"}
Sep 06 19:16:15.456131 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:15.456118    2581 scope.go:117] "RemoveContainer" containerID="e6f70f18cf94611867449a0d36c5d0a9d58c642c0c3c3cbec810e78e12a37bdb"
Sep 06 19:16:16.461859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:16.461820    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-w6lgm" event={"ID":"e97e47da-6bc6-48f8-bf89-18da9c037980","Type":"ContainerStarted","Data":"a6e94ca501adb58f9b12ead03af3dbc5b001dd3c4a734b046dac766f2dac93a3"}
Sep 06 19:16:18.026380 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.026303    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64cb479944-7zgtv" podUID="b037e792-6637-4021-9737-e2305570c10b" containerName="console" containerID="cri-o://f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287" gracePeriod=15
Sep 06 19:16:18.270832 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.270803    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:16:18.427904 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.427869    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-serving-cert\") pod \"b037e792-6637-4021-9737-e2305570c10b\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") "
Sep 06 19:16:18.428128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.427916    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-console-config\") pod \"b037e792-6637-4021-9737-e2305570c10b\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") "
Sep 06 19:16:18.428128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.427951    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5485x\" (UniqueName: \"kubernetes.io/projected/b037e792-6637-4021-9737-e2305570c10b-kube-api-access-5485x\") pod \"b037e792-6637-4021-9737-e2305570c10b\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") "
Sep 06 19:16:18.428128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.427966    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-service-ca\") pod \"b037e792-6637-4021-9737-e2305570c10b\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") "
Sep 06 19:16:18.428128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.428002    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-oauth-config\") pod \"b037e792-6637-4021-9737-e2305570c10b\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") "
Sep 06 19:16:18.428128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.428108    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-oauth-serving-cert\") pod \"b037e792-6637-4021-9737-e2305570c10b\" (UID: \"b037e792-6637-4021-9737-e2305570c10b\") "
Sep 06 19:16:18.428519 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.428488    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-service-ca" (OuterVolumeSpecName: "service-ca") pod "b037e792-6637-4021-9737-e2305570c10b" (UID: "b037e792-6637-4021-9737-e2305570c10b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:16:18.428519 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.428501    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-console-config" (OuterVolumeSpecName: "console-config") pod "b037e792-6637-4021-9737-e2305570c10b" (UID: "b037e792-6637-4021-9737-e2305570c10b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:16:18.428672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.428595    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b037e792-6637-4021-9737-e2305570c10b" (UID: "b037e792-6637-4021-9737-e2305570c10b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:16:18.430538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.430514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b037e792-6637-4021-9737-e2305570c10b" (UID: "b037e792-6637-4021-9737-e2305570c10b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:16:18.430538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.430531    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b037e792-6637-4021-9737-e2305570c10b" (UID: "b037e792-6637-4021-9737-e2305570c10b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:16:18.430687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.430515    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b037e792-6637-4021-9737-e2305570c10b-kube-api-access-5485x" (OuterVolumeSpecName: "kube-api-access-5485x") pod "b037e792-6637-4021-9737-e2305570c10b" (UID: "b037e792-6637-4021-9737-e2305570c10b"). InnerVolumeSpecName "kube-api-access-5485x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:16:18.472574 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.472536    2581 generic.go:356] "Generic (PLEG): container finished" podID="b037e792-6637-4021-9737-e2305570c10b" containerID="f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287" exitCode=0
Sep 06 19:16:18.472738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.472587    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cb479944-7zgtv"
Sep 06 19:16:18.472738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.472631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cb479944-7zgtv" event={"ID":"b037e792-6637-4021-9737-e2305570c10b","Type":"ContainerDied","Data":"f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287"}
Sep 06 19:16:18.472738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.472687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cb479944-7zgtv" event={"ID":"b037e792-6637-4021-9737-e2305570c10b","Type":"ContainerDied","Data":"d3bab8d489cac29298767202041c0f40b6bd63f0e52ade729dadac104f7cf553"}
Sep 06 19:16:18.472738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.472709    2581 scope.go:117] "RemoveContainer" containerID="f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287"
Sep 06 19:16:18.481643 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.481620    2581 scope.go:117] "RemoveContainer" containerID="f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287"
Sep 06 19:16:18.481935 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:16:18.481918    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287\": container with ID starting with f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287 not found: ID does not exist" containerID="f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287"
Sep 06 19:16:18.481983 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.481944    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287"} err="failed to get container status \"f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287\": rpc error: code = NotFound desc = could not find container \"f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287\": container with ID starting with f115af3ebb27e161364d2c0fa8c07eb163d1a216d27800bad2cd3061b7f7d287 not found: ID does not exist"
Sep 06 19:16:18.491650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.491617    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64cb479944-7zgtv"]
Sep 06 19:16:18.495384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.495360    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64cb479944-7zgtv"]
Sep 06 19:16:18.529090 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.529029    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-console-config\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:18.529254 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.529098    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5485x\" (UniqueName: \"kubernetes.io/projected/b037e792-6637-4021-9737-e2305570c10b-kube-api-access-5485x\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:18.529254 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.529113    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:18.529254 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.529123    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-oauth-config\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:18.529254 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.529131    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b037e792-6637-4021-9737-e2305570c10b-oauth-serving-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:18.529254 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:18.529141    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b037e792-6637-4021-9737-e2305570c10b-console-serving-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:20.264905 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:20.264872    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b037e792-6637-4021-9737-e2305570c10b" path="/var/lib/kubelet/pods/b037e792-6637-4021-9737-e2305570c10b/volumes"
Sep 06 19:16:20.514335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:20.514299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:20.611631 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:20.611557    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:24.499895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:24.499865    2581 generic.go:356] "Generic (PLEG): container finished" podID="869540bb-a576-4edf-9c8c-0aaa8c64ccc9" containerID="89b57d4cfca1eff096a044632ad158b45ddd60e0358e7e6835d2fb7317479dd2" exitCode=0
Sep 06 19:16:24.500319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:24.499940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6" event={"ID":"869540bb-a576-4edf-9c8c-0aaa8c64ccc9","Type":"ContainerDied","Data":"89b57d4cfca1eff096a044632ad158b45ddd60e0358e7e6835d2fb7317479dd2"}
Sep 06 19:16:24.500685 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:24.500667    2581 scope.go:117] "RemoveContainer" containerID="89b57d4cfca1eff096a044632ad158b45ddd60e0358e7e6835d2fb7317479dd2"
Sep 06 19:16:25.505849 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:25.505813    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-vnqh6" event={"ID":"869540bb-a576-4edf-9c8c-0aaa8c64ccc9","Type":"ContainerStarted","Data":"70a272a6d8342e20c793df21f14d25859c609b41e7259163f3c0c2bfea4032cb"}
Sep 06 19:16:27.847862 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:27.847834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:16:27.851929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:27.851903    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6d998fc79-f5pnk"
Sep 06 19:16:29.310756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.310720    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fdb697ff4-7776p" podUID="98520c34-904c-472d-a746-c57d372ebd89" containerName="console" containerID="cri-o://da308ad6780b89ea6b9bf7a6577d34f4593f27c72690755bc03c89b629753b36" gracePeriod=15
Sep 06 19:16:29.532207 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.532173    2581 generic.go:356] "Generic (PLEG): container finished" podID="98520c34-904c-472d-a746-c57d372ebd89" containerID="da308ad6780b89ea6b9bf7a6577d34f4593f27c72690755bc03c89b629753b36" exitCode=0
Sep 06 19:16:29.532373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.532254    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb697ff4-7776p" event={"ID":"98520c34-904c-472d-a746-c57d372ebd89","Type":"ContainerDied","Data":"da308ad6780b89ea6b9bf7a6577d34f4593f27c72690755bc03c89b629753b36"}
Sep 06 19:16:29.575784 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.575751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:29.594412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.594375    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:16:29.618746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.618700    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-service-ca\") pod \"98520c34-904c-472d-a746-c57d372ebd89\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") "
Sep 06 19:16:29.618746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.618740    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-oauth-serving-cert\") pod \"98520c34-904c-472d-a746-c57d372ebd89\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") "
Sep 06 19:16:29.618991 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.618788    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-serving-cert\") pod \"98520c34-904c-472d-a746-c57d372ebd89\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") "
Sep 06 19:16:29.618991 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.618812    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-console-config\") pod \"98520c34-904c-472d-a746-c57d372ebd89\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") "
Sep 06 19:16:29.618991 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.618891    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-oauth-config\") pod \"98520c34-904c-472d-a746-c57d372ebd89\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") "
Sep 06 19:16:29.618991 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.618915    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-trusted-ca-bundle\") pod \"98520c34-904c-472d-a746-c57d372ebd89\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") "
Sep 06 19:16:29.618991 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.618948    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf9qb\" (UniqueName: \"kubernetes.io/projected/98520c34-904c-472d-a746-c57d372ebd89-kube-api-access-pf9qb\") pod \"98520c34-904c-472d-a746-c57d372ebd89\" (UID: \"98520c34-904c-472d-a746-c57d372ebd89\") "
Sep 06 19:16:29.619273 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.619155    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-service-ca" (OuterVolumeSpecName: "service-ca") pod "98520c34-904c-472d-a746-c57d372ebd89" (UID: "98520c34-904c-472d-a746-c57d372ebd89"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:16:29.619273 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.619243    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:29.619374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.619283    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-console-config" (OuterVolumeSpecName: "console-config") pod "98520c34-904c-472d-a746-c57d372ebd89" (UID: "98520c34-904c-472d-a746-c57d372ebd89"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:16:29.619374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.619291    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "98520c34-904c-472d-a746-c57d372ebd89" (UID: "98520c34-904c-472d-a746-c57d372ebd89"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:16:29.619624 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.619602    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "98520c34-904c-472d-a746-c57d372ebd89" (UID: "98520c34-904c-472d-a746-c57d372ebd89"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:16:29.621329 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.621300    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "98520c34-904c-472d-a746-c57d372ebd89" (UID: "98520c34-904c-472d-a746-c57d372ebd89"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:16:29.621464 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.621447    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "98520c34-904c-472d-a746-c57d372ebd89" (UID: "98520c34-904c-472d-a746-c57d372ebd89"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:16:29.621657 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.621628    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98520c34-904c-472d-a746-c57d372ebd89-kube-api-access-pf9qb" (OuterVolumeSpecName: "kube-api-access-pf9qb") pod "98520c34-904c-472d-a746-c57d372ebd89" (UID: "98520c34-904c-472d-a746-c57d372ebd89"). InnerVolumeSpecName "kube-api-access-pf9qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:16:29.677218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.677184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:16:29.720297 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.720256    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-serving-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:29.720297 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.720291    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-console-config\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:29.720297 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.720306    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98520c34-904c-472d-a746-c57d372ebd89-console-oauth-config\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:29.720539 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.720318    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-trusted-ca-bundle\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:29.720539 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.720332    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf9qb\" (UniqueName: \"kubernetes.io/projected/98520c34-904c-472d-a746-c57d372ebd89-kube-api-access-pf9qb\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:29.720539 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:29.720344    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98520c34-904c-472d-a746-c57d372ebd89-oauth-serving-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:16:30.538108 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:30.538015    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb697ff4-7776p"
Sep 06 19:16:30.538108 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:30.538026    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb697ff4-7776p" event={"ID":"98520c34-904c-472d-a746-c57d372ebd89","Type":"ContainerDied","Data":"9b9bbabda1ea1103414bc1b6e796be739421124c22eb8404c24f9a182b3f2366"}
Sep 06 19:16:30.538629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:30.538128    2581 scope.go:117] "RemoveContainer" containerID="da308ad6780b89ea6b9bf7a6577d34f4593f27c72690755bc03c89b629753b36"
Sep 06 19:16:30.554581 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:30.554549    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fdb697ff4-7776p"]
Sep 06 19:16:30.555947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:30.555927    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fdb697ff4-7776p"]
Sep 06 19:16:32.264420 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:32.264385    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98520c34-904c-472d-a746-c57d372ebd89" path="/var/lib/kubelet/pods/98520c34-904c-472d-a746-c57d372ebd89/volumes"
Sep 06 19:16:49.608041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:49.608008    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:16:49.624667 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:49.624627    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:16:49.640733 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:16:49.640710    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:02.269918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:02.269886    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:02.667214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:02.667155    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:07.631513 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.631423    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 19:17:07.631968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.631821    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="prometheus" containerID="cri-o://ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658" gracePeriod=600
Sep 06 19:17:07.631968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.631848    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy" containerID="cri-o://707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae" gracePeriod=600
Sep 06 19:17:07.631968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.631872    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="config-reloader" containerID="cri-o://ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0" gracePeriod=600
Sep 06 19:17:07.631968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.631903    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy-thanos" containerID="cri-o://f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7" gracePeriod=600
Sep 06 19:17:07.631968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.631863    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="thanos-sidecar" containerID="cri-o://119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858" gracePeriod=600
Sep 06 19:17:07.632240 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.631939    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy-web" containerID="cri-o://20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7" gracePeriod=600
Sep 06 19:17:07.871906 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.871877    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:07.953341 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953311    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgrsq\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-kube-api-access-hgrsq\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953341 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953345    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-config-out\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953380    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-2\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953410    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-trusted-ca-bundle\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953450    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-grpc-tls\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953477    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953502    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-metrics-client-certs\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953525    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-kube-rbac-proxy\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953552    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-tls\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953584    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-1\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953611    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-tls-assets\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-config\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953671    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-db\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953696    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-0\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.953929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953720    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-web-config\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.954267 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.953928    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:17:07.954267 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.954149    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-metrics-client-ca\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.954384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.954299    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-serving-certs-ca-bundle\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.954384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.954338    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-thanos-prometheus-http-client-file\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.954384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.954369    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.954544 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.954413    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-kubelet-serving-ca-bundle\") pod \"186bcb5f-0915-4769-acc3-34a69b334423\" (UID: \"186bcb5f-0915-4769-acc3-34a69b334423\") "
Sep 06 19:17:07.954690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.954662    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:17:07.955082 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.955042    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:17:07.955582 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.955552    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:17:07.956660 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.956609    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:17:07.957190 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.956890    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:17:07.957507 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957482    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-config-out" (OuterVolumeSpecName: "config-out") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:17:07.957590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957490    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:07.957590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957524    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:17:07.957702 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957602    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:07.957702 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957622    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:07.957969 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957708    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-db\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:07.957969 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957816    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-metrics-client-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:07.957969 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957831    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:07.957969 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957916    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:17:07.958200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957972    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-kube-api-access-hgrsq" (OuterVolumeSpecName: "kube-api-access-hgrsq") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "kube-api-access-hgrsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:17:07.958200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.957975    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:17:07.958200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.958177    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-config" (OuterVolumeSpecName: "config") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:17:07.959873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.959838    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:17:07.959987 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.959957    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:17:07.960050 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.959986    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:17:07.960050 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.960019    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:17:07.960199 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.960173    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:17:07.960308 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.960271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:17:07.960417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.960400    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:17:07.970767 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:07.970736    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-web-config" (OuterVolumeSpecName: "web-config") pod "186bcb5f-0915-4769-acc3-34a69b334423" (UID: "186bcb5f-0915-4769-acc3-34a69b334423"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:17:08.058612 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058572    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-grpc-tls\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058612 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058606    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058612 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058620    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-metrics-client-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058633    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-kube-rbac-proxy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058645    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058656    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-tls-assets\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058666    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-config\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058676    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058686    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-web-config\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058705    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058718    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/186bcb5f-0915-4769-acc3-34a69b334423-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058730    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186bcb5f-0915-4769-acc3-34a69b334423-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058741    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgrsq\" (UniqueName: \"kubernetes.io/projected/186bcb5f-0915-4769-acc3-34a69b334423-kube-api-access-hgrsq\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.058866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.058753    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/186bcb5f-0915-4769-acc3-34a69b334423-config-out\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:17:08.692154 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692118    2581 generic.go:356] "Generic (PLEG): container finished" podID="186bcb5f-0915-4769-acc3-34a69b334423" containerID="f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7" exitCode=0
Sep 06 19:17:08.692154 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692147    2581 generic.go:356] "Generic (PLEG): container finished" podID="186bcb5f-0915-4769-acc3-34a69b334423" containerID="707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae" exitCode=0
Sep 06 19:17:08.692154 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692153    2581 generic.go:356] "Generic (PLEG): container finished" podID="186bcb5f-0915-4769-acc3-34a69b334423" containerID="20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7" exitCode=0
Sep 06 19:17:08.692154 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692160    2581 generic.go:356] "Generic (PLEG): container finished" podID="186bcb5f-0915-4769-acc3-34a69b334423" containerID="119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858" exitCode=0
Sep 06 19:17:08.692154 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692165    2581 generic.go:356] "Generic (PLEG): container finished" podID="186bcb5f-0915-4769-acc3-34a69b334423" containerID="ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0" exitCode=0
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692170    2581 generic.go:356] "Generic (PLEG): container finished" podID="186bcb5f-0915-4769-acc3-34a69b334423" containerID="ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658" exitCode=0
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692216    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692216    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerDied","Data":"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"}
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692291    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerDied","Data":"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"}
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerDied","Data":"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"}
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692342    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerDied","Data":"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"}
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692356    2581 scope.go:117] "RemoveContainer" containerID="f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerDied","Data":"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"}
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerDied","Data":"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"}
Sep 06 19:17:08.692707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.692432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"186bcb5f-0915-4769-acc3-34a69b334423","Type":"ContainerDied","Data":"17eede25ea368687df6a119937db526ecfb85f3c6f04ea0886804adc7200244d"}
Sep 06 19:17:08.700517 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.700500    2581 scope.go:117] "RemoveContainer" containerID="707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"
Sep 06 19:17:08.707794 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.707775    2581 scope.go:117] "RemoveContainer" containerID="20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"
Sep 06 19:17:08.712373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.712348    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 19:17:08.715468 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.715447    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 19:17:08.715923 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.715909    2581 scope.go:117] "RemoveContainer" containerID="119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"
Sep 06 19:17:08.723009 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.722990    2581 scope.go:117] "RemoveContainer" containerID="ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"
Sep 06 19:17:08.730261 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.730241    2581 scope.go:117] "RemoveContainer" containerID="ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"
Sep 06 19:17:08.738404 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.738379    2581 scope.go:117] "RemoveContainer" containerID="561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"
Sep 06 19:17:08.738783 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.738765    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 19:17:08.739122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739101    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy-web"
Sep 06 19:17:08.739122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739124    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy-web"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739138    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy-thanos"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739147    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy-thanos"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739158    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739166    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739189    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98520c34-904c-472d-a746-c57d372ebd89" containerName="console"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739194    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="98520c34-904c-472d-a746-c57d372ebd89" containerName="console"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739200    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="config-reloader"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739205    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="config-reloader"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739213    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b037e792-6637-4021-9737-e2305570c10b" containerName="console"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739218    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b037e792-6637-4021-9737-e2305570c10b" containerName="console"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739232    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="init-config-reloader"
Sep 06 19:17:08.739237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739240    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="init-config-reloader"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739250    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="prometheus"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739259    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="prometheus"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739267    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="thanos-sidecar"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739276    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="thanos-sidecar"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739367    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="prometheus"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739379    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy-thanos"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739387    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="98520c34-904c-472d-a746-c57d372ebd89" containerName="console"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739396    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="config-reloader"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739405    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739413    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="thanos-sidecar"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739422    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="186bcb5f-0915-4769-acc3-34a69b334423" containerName="kube-rbac-proxy-web"
Sep 06 19:17:08.739629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.739431    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b037e792-6637-4021-9737-e2305570c10b" containerName="console"
Sep 06 19:17:08.744866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.744848    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.746207 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.746187    2581 scope.go:117] "RemoveContainer" containerID="f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"
Sep 06 19:17:08.746507 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:17:08.746487    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": container with ID starting with f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7 not found: ID does not exist" containerID="f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"
Sep 06 19:17:08.746583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.746519    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"} err="failed to get container status \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": rpc error: code = NotFound desc = could not find container \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": container with ID starting with f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7 not found: ID does not exist"
Sep 06 19:17:08.746583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.746543    2581 scope.go:117] "RemoveContainer" containerID="707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"
Sep 06 19:17:08.746866 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:17:08.746844    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": container with ID starting with 707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae not found: ID does not exist" containerID="707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"
Sep 06 19:17:08.746957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.746875    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"} err="failed to get container status \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": rpc error: code = NotFound desc = could not find container \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": container with ID starting with 707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae not found: ID does not exist"
Sep 06 19:17:08.746957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.746895    2581 scope.go:117] "RemoveContainer" containerID="20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"
Sep 06 19:17:08.747198 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:17:08.747149    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": container with ID starting with 20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7 not found: ID does not exist" containerID="20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"
Sep 06 19:17:08.747198 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747181    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"} err="failed to get container status \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": rpc error: code = NotFound desc = could not find container \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": container with ID starting with 20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7 not found: ID does not exist"
Sep 06 19:17:08.747453 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747202    2581 scope.go:117] "RemoveContainer" containerID="119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"
Sep 06 19:17:08.747453 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747252    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4ogclut4d2o01\""
Sep 06 19:17:08.747453 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747315    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 06 19:17:08.747453 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747325    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 06 19:17:08.747453 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747349    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 06 19:17:08.747453 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747380    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 06 19:17:08.747453 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747389    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m2xbg\""
Sep 06 19:17:08.747945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747597    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 06 19:17:08.747945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747645    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 06 19:17:08.747945 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:17:08.747794    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": container with ID starting with 119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858 not found: ID does not exist" containerID="119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"
Sep 06 19:17:08.747945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747823    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"} err="failed to get container status \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": rpc error: code = NotFound desc = could not find container \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": container with ID starting with 119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858 not found: ID does not exist"
Sep 06 19:17:08.747945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747843    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 06 19:17:08.747945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747871    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 06 19:17:08.747945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747844    2581 scope.go:117] "RemoveContainer" containerID="ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"
Sep 06 19:17:08.748415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.747995    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 06 19:17:08.748415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.748218    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 06 19:17:08.748415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.748223    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 06 19:17:08.748415 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:17:08.748285    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": container with ID starting with ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0 not found: ID does not exist" containerID="ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"
Sep 06 19:17:08.748415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.748318    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"} err="failed to get container status \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": rpc error: code = NotFound desc = could not find container \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": container with ID starting with ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0 not found: ID does not exist"
Sep 06 19:17:08.748415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.748340    2581 scope.go:117] "RemoveContainer" containerID="ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"
Sep 06 19:17:08.748993 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.748481    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 06 19:17:08.748993 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:17:08.748869    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": container with ID starting with ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658 not found: ID does not exist" containerID="ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"
Sep 06 19:17:08.748993 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.748897    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"} err="failed to get container status \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": rpc error: code = NotFound desc = could not find container \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": container with ID starting with ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658 not found: ID does not exist"
Sep 06 19:17:08.748993 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.748917    2581 scope.go:117] "RemoveContainer" containerID="561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"
Sep 06 19:17:08.749543 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:17:08.749199    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": container with ID starting with 561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b not found: ID does not exist" containerID="561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"
Sep 06 19:17:08.749543 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.749225    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"} err="failed to get container status \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": rpc error: code = NotFound desc = could not find container \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": container with ID starting with 561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b not found: ID does not exist"
Sep 06 19:17:08.749543 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.749238    2581 scope.go:117] "RemoveContainer" containerID="f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"
Sep 06 19:17:08.749760 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.749568    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"} err="failed to get container status \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": rpc error: code = NotFound desc = could not find container \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": container with ID starting with f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7 not found: ID does not exist"
Sep 06 19:17:08.749760 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.749585    2581 scope.go:117] "RemoveContainer" containerID="707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"
Sep 06 19:17:08.749982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.749844    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"} err="failed to get container status \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": rpc error: code = NotFound desc = could not find container \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": container with ID starting with 707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae not found: ID does not exist"
Sep 06 19:17:08.749982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.749860    2581 scope.go:117] "RemoveContainer" containerID="20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"
Sep 06 19:17:08.750218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.750193    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"} err="failed to get container status \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": rpc error: code = NotFound desc = could not find container \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": container with ID starting with 20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7 not found: ID does not exist"
Sep 06 19:17:08.750218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.750220    2581 scope.go:117] "RemoveContainer" containerID="119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"
Sep 06 19:17:08.750442 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.750424    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 06 19:17:08.750564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.750532    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"} err="failed to get container status \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": rpc error: code = NotFound desc = could not find container \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": container with ID starting with 119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858 not found: ID does not exist"
Sep 06 19:17:08.750564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.750554    2581 scope.go:117] "RemoveContainer" containerID="ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"
Sep 06 19:17:08.750797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.750766    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"} err="failed to get container status \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": rpc error: code = NotFound desc = could not find container \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": container with ID starting with ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0 not found: ID does not exist"
Sep 06 19:17:08.750797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.750786    2581 scope.go:117] "RemoveContainer" containerID="ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"
Sep 06 19:17:08.751047 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.751018    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"} err="failed to get container status \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": rpc error: code = NotFound desc = could not find container \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": container with ID starting with ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658 not found: ID does not exist"
Sep 06 19:17:08.751047 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.751035    2581 scope.go:117] "RemoveContainer" containerID="561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"
Sep 06 19:17:08.751295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.751271    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"} err="failed to get container status \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": rpc error: code = NotFound desc = could not find container \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": container with ID starting with 561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b not found: ID does not exist"
Sep 06 19:17:08.751295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.751296    2581 scope.go:117] "RemoveContainer" containerID="f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"
Sep 06 19:17:08.751604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.751585    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"} err="failed to get container status \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": rpc error: code = NotFound desc = could not find container \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": container with ID starting with f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7 not found: ID does not exist"
Sep 06 19:17:08.751604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.751604    2581 scope.go:117] "RemoveContainer" containerID="707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"
Sep 06 19:17:08.751898 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.751872    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"} err="failed to get container status \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": rpc error: code = NotFound desc = could not find container \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": container with ID starting with 707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae not found: ID does not exist"
Sep 06 19:17:08.751898 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.751898    2581 scope.go:117] "RemoveContainer" containerID="20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"
Sep 06 19:17:08.752275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.752250    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"} err="failed to get container status \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": rpc error: code = NotFound desc = could not find container \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": container with ID starting with 20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7 not found: ID does not exist"
Sep 06 19:17:08.752388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.752275    2581 scope.go:117] "RemoveContainer" containerID="119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"
Sep 06 19:17:08.752562 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.752542    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"} err="failed to get container status \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": rpc error: code = NotFound desc = could not find container \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": container with ID starting with 119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858 not found: ID does not exist"
Sep 06 19:17:08.752562 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.752561    2581 scope.go:117] "RemoveContainer" containerID="ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"
Sep 06 19:17:08.752835 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.752811    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"} err="failed to get container status \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": rpc error: code = NotFound desc = could not find container \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": container with ID starting with ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0 not found: ID does not exist"
Sep 06 19:17:08.752901 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.752838    2581 scope.go:117] "RemoveContainer" containerID="ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"
Sep 06 19:17:08.752901 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.752865    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 06 19:17:08.753284 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.753109    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"} err="failed to get container status \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": rpc error: code = NotFound desc = could not find container \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": container with ID starting with ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658 not found: ID does not exist"
Sep 06 19:17:08.753284 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.753128    2581 scope.go:117] "RemoveContainer" containerID="561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"
Sep 06 19:17:08.753482 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.753460    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"} err="failed to get container status \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": rpc error: code = NotFound desc = could not find container \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": container with ID starting with 561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b not found: ID does not exist"
Sep 06 19:17:08.753551 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.753485    2581 scope.go:117] "RemoveContainer" containerID="f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"
Sep 06 19:17:08.753760 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.753736    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"} err="failed to get container status \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": rpc error: code = NotFound desc = could not find container \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": container with ID starting with f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7 not found: ID does not exist"
Sep 06 19:17:08.753898 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.753762    2581 scope.go:117] "RemoveContainer" containerID="707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"
Sep 06 19:17:08.754020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.754000    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"} err="failed to get container status \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": rpc error: code = NotFound desc = could not find container \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": container with ID starting with 707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae not found: ID does not exist"
Sep 06 19:17:08.754020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.754019    2581 scope.go:117] "RemoveContainer" containerID="20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"
Sep 06 19:17:08.754349 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.754324    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"} err="failed to get container status \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": rpc error: code = NotFound desc = could not find container \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": container with ID starting with 20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7 not found: ID does not exist"
Sep 06 19:17:08.754349 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.754349    2581 scope.go:117] "RemoveContainer" containerID="119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"
Sep 06 19:17:08.755045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.755004    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"} err="failed to get container status \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": rpc error: code = NotFound desc = could not find container \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": container with ID starting with 119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858 not found: ID does not exist"
Sep 06 19:17:08.755175 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.755048    2581 scope.go:117] "RemoveContainer" containerID="ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"
Sep 06 19:17:08.755777 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.755741    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"} err="failed to get container status \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": rpc error: code = NotFound desc = could not find container \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": container with ID starting with ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0 not found: ID does not exist"
Sep 06 19:17:08.755866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.755780    2581 scope.go:117] "RemoveContainer" containerID="ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"
Sep 06 19:17:08.756082 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.756045    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"} err="failed to get container status \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": rpc error: code = NotFound desc = could not find container \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": container with ID starting with ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658 not found: ID does not exist"
Sep 06 19:17:08.756168 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.756084    2581 scope.go:117] "RemoveContainer" containerID="561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"
Sep 06 19:17:08.756415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.756396    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"} err="failed to get container status \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": rpc error: code = NotFound desc = could not find container \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": container with ID starting with 561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b not found: ID does not exist"
Sep 06 19:17:08.756415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.756415    2581 scope.go:117] "RemoveContainer" containerID="f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"
Sep 06 19:17:08.756693 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.756677    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"} err="failed to get container status \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": rpc error: code = NotFound desc = could not find container \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": container with ID starting with f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7 not found: ID does not exist"
Sep 06 19:17:08.756746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.756694    2581 scope.go:117] "RemoveContainer" containerID="707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"
Sep 06 19:17:08.756827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.756809    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 19:17:08.756954 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.756938    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"} err="failed to get container status \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": rpc error: code = NotFound desc = could not find container \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": container with ID starting with 707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae not found: ID does not exist"
Sep 06 19:17:08.757086 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.756954    2581 scope.go:117] "RemoveContainer" containerID="20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"
Sep 06 19:17:08.757241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.757225    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"} err="failed to get container status \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": rpc error: code = NotFound desc = could not find container \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": container with ID starting with 20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7 not found: ID does not exist"
Sep 06 19:17:08.757285 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.757243    2581 scope.go:117] "RemoveContainer" containerID="119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"
Sep 06 19:17:08.757499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.757483    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"} err="failed to get container status \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": rpc error: code = NotFound desc = could not find container \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": container with ID starting with 119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858 not found: ID does not exist"
Sep 06 19:17:08.757499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.757499    2581 scope.go:117] "RemoveContainer" containerID="ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"
Sep 06 19:17:08.757726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.757701    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"} err="failed to get container status \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": rpc error: code = NotFound desc = could not find container \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": container with ID starting with ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0 not found: ID does not exist"
Sep 06 19:17:08.757807 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.757726    2581 scope.go:117] "RemoveContainer" containerID="ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"
Sep 06 19:17:08.757974 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.757958    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"} err="failed to get container status \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": rpc error: code = NotFound desc = could not find container \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": container with ID starting with ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658 not found: ID does not exist"
Sep 06 19:17:08.758022 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.757975    2581 scope.go:117] "RemoveContainer" containerID="561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"
Sep 06 19:17:08.758221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.758204    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"} err="failed to get container status \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": rpc error: code = NotFound desc = could not find container \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": container with ID starting with 561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b not found: ID does not exist"
Sep 06 19:17:08.758221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.758220    2581 scope.go:117] "RemoveContainer" containerID="f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"
Sep 06 19:17:08.758463 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.758442    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7"} err="failed to get container status \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": rpc error: code = NotFound desc = could not find container \"f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7\": container with ID starting with f2fea6adc0f62acd932b19ab07acf6b1580c2185701cb362fdc3eff4432ff2e7 not found: ID does not exist"
Sep 06 19:17:08.758515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.758464    2581 scope.go:117] "RemoveContainer" containerID="707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"
Sep 06 19:17:08.758753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.758733    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae"} err="failed to get container status \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": rpc error: code = NotFound desc = could not find container \"707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae\": container with ID starting with 707dc122f9148e04dcc1e559b214def57574ad943f99facaad968a15fc9741ae not found: ID does not exist"
Sep 06 19:17:08.758835 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.758755    2581 scope.go:117] "RemoveContainer" containerID="20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"
Sep 06 19:17:08.759078 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.759040    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7"} err="failed to get container status \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": rpc error: code = NotFound desc = could not find container \"20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7\": container with ID starting with 20dad2b946b5e4acdd521a892e47338c0eccdc8baf89404dae4d866b90407bb7 not found: ID does not exist"
Sep 06 19:17:08.759168 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.759083    2581 scope.go:117] "RemoveContainer" containerID="119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"
Sep 06 19:17:08.759395 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.759359    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858"} err="failed to get container status \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": rpc error: code = NotFound desc = could not find container \"119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858\": container with ID starting with 119268204c552ec7f5d6e040eb748f154ccf3df4fc9222774853c43ae4cb8858 not found: ID does not exist"
Sep 06 19:17:08.759395 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.759380    2581 scope.go:117] "RemoveContainer" containerID="ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"
Sep 06 19:17:08.759678 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.759660    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0"} err="failed to get container status \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": rpc error: code = NotFound desc = could not find container \"ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0\": container with ID starting with ba95485b506082cba32d12dcfefc0d4abb91b49e5a093e2fc211441497cfa3f0 not found: ID does not exist"
Sep 06 19:17:08.759760 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.759679    2581 scope.go:117] "RemoveContainer" containerID="ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"
Sep 06 19:17:08.759928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.759911    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658"} err="failed to get container status \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": rpc error: code = NotFound desc = could not find container \"ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658\": container with ID starting with ace77f2e8148020cd5b21ee9b659bcacd77fa8e1bd26a43a32477887a003c658 not found: ID does not exist"
Sep 06 19:17:08.759976 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.759929    2581 scope.go:117] "RemoveContainer" containerID="561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"
Sep 06 19:17:08.760174 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.760152    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b"} err="failed to get container status \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": rpc error: code = NotFound desc = could not find container \"561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b\": container with ID starting with 561dce604ce16b3013180580281d3ca34388140082c434d6da75246e2e54097b not found: ID does not exist"
Sep 06 19:17:08.765521 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trznp\" (UniqueName: \"kubernetes.io/projected/94996cdc-40b3-4739-a8cf-7a707ff38c12-kube-api-access-trznp\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765693 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765693 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765693 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765801 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765801 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-config\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765801 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765801 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/94996cdc-40b3-4739-a8cf-7a707ff38c12-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765801 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765783    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765943 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765943 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765943 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765891    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765943 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/94996cdc-40b3-4739-a8cf-7a707ff38c12-config-out\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.765943 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.766135 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.765979    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-web-config\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.766135 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.766001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.766135 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.766027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.766135 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.766083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867371 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867371 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867371 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trznp\" (UniqueName: \"kubernetes.io/projected/94996cdc-40b3-4739-a8cf-7a707ff38c12-kube-api-access-trznp\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-config\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867660    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.867690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/94996cdc-40b3-4739-a8cf-7a707ff38c12-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867719    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/94996cdc-40b3-4739-a8cf-7a707ff38c12-config-out\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-web-config\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.867990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.868027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.868318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868627 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.868577    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.868829 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.868656    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.869458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.869425    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.871590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.870925    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.871590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.870971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.871590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.871112    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-config\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.871590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.871167    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.871590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.871293    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-web-config\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.871590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.871469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.871971 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.871659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.871971 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.871792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.871971 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.871913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/94996cdc-40b3-4739-a8cf-7a707ff38c12-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.873138 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.872729    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/94996cdc-40b3-4739-a8cf-7a707ff38c12-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.873687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.873661    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.874127 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.874095    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.874338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.874310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.874427 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.874361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/94996cdc-40b3-4739-a8cf-7a707ff38c12-config-out\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.874482 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.874460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/94996cdc-40b3-4739-a8cf-7a707ff38c12-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:08.875325 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:08.875307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trznp\" (UniqueName: \"kubernetes.io/projected/94996cdc-40b3-4739-a8cf-7a707ff38c12-kube-api-access-trznp\") pod \"prometheus-k8s-0\" (UID: \"94996cdc-40b3-4739-a8cf-7a707ff38c12\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:09.057767 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:09.057683    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:09.339682 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:09.338002    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 19:17:09.342483 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:17:09.342453    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94996cdc_40b3_4739_a8cf_7a707ff38c12.slice/crio-b3df70f5a65002a5a2455e2b0c198d98b1baed1e7c5ec5cb5fe1fb5c218aa862 WatchSource:0}: Error finding container b3df70f5a65002a5a2455e2b0c198d98b1baed1e7c5ec5cb5fe1fb5c218aa862: Status 404 returned error can't find the container with id b3df70f5a65002a5a2455e2b0c198d98b1baed1e7c5ec5cb5fe1fb5c218aa862
Sep 06 19:17:09.698140 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:09.698110    2581 generic.go:356] "Generic (PLEG): container finished" podID="94996cdc-40b3-4739-a8cf-7a707ff38c12" containerID="2cc524c952cc14a836bbbe533a0aaaa07fe62b186bf561f0846d74cc1d58fddf" exitCode=0
Sep 06 19:17:09.698556 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:09.698162    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94996cdc-40b3-4739-a8cf-7a707ff38c12","Type":"ContainerDied","Data":"2cc524c952cc14a836bbbe533a0aaaa07fe62b186bf561f0846d74cc1d58fddf"}
Sep 06 19:17:09.698556 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:09.698214    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94996cdc-40b3-4739-a8cf-7a707ff38c12","Type":"ContainerStarted","Data":"b3df70f5a65002a5a2455e2b0c198d98b1baed1e7c5ec5cb5fe1fb5c218aa862"}
Sep 06 19:17:10.264480 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.264394    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="186bcb5f-0915-4769-acc3-34a69b334423" path="/var/lib/kubelet/pods/186bcb5f-0915-4769-acc3-34a69b334423/volumes"
Sep 06 19:17:10.647587 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.647547    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:10.709444 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.709408    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94996cdc-40b3-4739-a8cf-7a707ff38c12","Type":"ContainerStarted","Data":"0b90b0c0382cb942d6dcf3a1856fed22e70d813d99cca24b9caa4527eb9c0368"}
Sep 06 19:17:10.709802 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.709469    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94996cdc-40b3-4739-a8cf-7a707ff38c12","Type":"ContainerStarted","Data":"e361f70bfad4a6a17eb7ec9afd7a761f350dacc1a133b19c98b0b334249b82ff"}
Sep 06 19:17:10.709802 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.709494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94996cdc-40b3-4739-a8cf-7a707ff38c12","Type":"ContainerStarted","Data":"e9f2cc00915bc03cfd162e967922ccfe40a860ba2c28d4dad6029669b6b1ca13"}
Sep 06 19:17:10.709802 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.709522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94996cdc-40b3-4739-a8cf-7a707ff38c12","Type":"ContainerStarted","Data":"f8ff525254209ed2e4ccaf922917d69bd08f9ce620f478c6d4981464e6ebf922"}
Sep 06 19:17:10.709802 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.709547    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94996cdc-40b3-4739-a8cf-7a707ff38c12","Type":"ContainerStarted","Data":"4ef11072a5c98c9fefc886011a2afdba68f64fb461e6c3509e00141495b6cfa2"}
Sep 06 19:17:10.709802 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.709569    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94996cdc-40b3-4739-a8cf-7a707ff38c12","Type":"ContainerStarted","Data":"a134ff09045e6d98a1bbac66d7ee81dfc1e92777d40bb51fe1aed03ee37d2cc0"}
Sep 06 19:17:10.735770 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.735709    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.716628451 podStartE2EDuration="2.735690481s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.06203ms" isStatefulPod=false podCreationTimestamp="2026-09-06 19:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:17:10.733535461 +0000 UTC m=+359.013988276" watchObservedRunningTime="2026-09-06 19:17:10.735690481 +0000 UTC m=+359.016143275"
Sep 06 19:17:10.800099 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.800033    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:10.899532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.899453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:10.948293 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:10.948257    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:11.397299 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:11.397273    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:11.496708 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:11.496673    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:14.058851 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:14.058816    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:17:21.700358 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:21.700320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:21.801809 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:21.801774    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:21.848074 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:21.848026    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:22.657679 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:22.657639    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:17:22.757993 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:17:22.757960    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:09.058168 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:09.058106    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:18:09.074697 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:09.074671    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:18:09.967104 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:09.967037    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 19:18:17.468226 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:17.468189    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:17.524680 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:17.524647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:26.177796 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:26.177760    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:26.225434 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:26.225396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:29.465267 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:29.465233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:29.512517 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:29.512480    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:36.993438 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:36.993403    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:37.043744 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:37.043703    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:37.099289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:37.099257    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:44.587305 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:44.587269    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:44.686056 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:44.686017    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:18:44.882876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:18:44.882851    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:11.307589 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:11.307556    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:29.407327 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:29.407290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:29.498531 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:29.498491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:50.741802 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:50.741769    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:52.094435 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:52.094401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:54.356731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:54.356694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:54.415424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:54.415387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:55.785449 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:55.785416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:55.883222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:55.883192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:56.584112 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:56.584075    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:56.679034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:56.679002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:57.983891 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:57.983843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:19:58.385206 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:19:58.385181    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:01.347757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:01.347721    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:01.410694 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:01.410663    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:02.003191 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:02.003159    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:05.509673 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:05.509582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:05.610546 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:05.610502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:05.793428 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:05.793345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:09.416374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:09.416342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:09.516253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:09.516218    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:15.016631 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:15.016594    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:15.116660 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:15.116622    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:16.771771 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:16.771736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:33.571442 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:33.571397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:33.671034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:33.670986    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:34.184252 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.184196    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:34.385906 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.385873    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:34.494797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.494697    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:34.573715 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.573669    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"]
Sep 06 19:20:34.577130 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.577102    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.579923 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.579898    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 06 19:20:34.580291 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.580273    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 06 19:20:34.580375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.580360    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 06 19:20:34.580738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.580724    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b4hv4\""
Sep 06 19:20:34.581290 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.581274    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 06 19:20:34.607581 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.607547    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"]
Sep 06 19:20:34.702056 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.702020    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:34.717816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.717785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vc84\" (UniqueName: \"kubernetes.io/projected/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-kube-api-access-8vc84\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.717924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.717830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-tmp\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.717924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.717850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-webhook-cert\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.718008 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.717954    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-manifests\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.718008 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.717981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.801494 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.801394    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:34.820503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.820458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-manifests\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.820668 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.820515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.820668 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.820588    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vc84\" (UniqueName: \"kubernetes.io/projected/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-kube-api-access-8vc84\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.820668 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.820630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-tmp\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.820668 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.820661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-webhook-cert\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.820967 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.820935    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-manifests\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.824132 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.824093    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-tmp\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.824388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.824370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.824541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.824515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-webhook-cert\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.829457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.829431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vc84\" (UniqueName: \"kubernetes.io/projected/8bfbac4f-4890-4bdb-9f21-72e6e476d78a-kube-api-access-8vc84\") pod \"opendatahub-operator-controller-manager-d4b9b44b9-x9qtc\" (UID: \"8bfbac4f-4890-4bdb-9f21-72e6e476d78a\") " pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.888180 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.888142    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:34.903998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:34.903952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:35.001502 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:35.001468    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:35.102592 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:35.102505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:35.175205 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:35.175170    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"]
Sep 06 19:20:35.178692 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:20:35.178563    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bfbac4f_4890_4bdb_9f21_72e6e476d78a.slice/crio-f0a6de2a4b1a3adf8cdd3bdafc40d4ec3543bd45a228383ff9a0aa1ccbedc8c4 WatchSource:0}: Error finding container f0a6de2a4b1a3adf8cdd3bdafc40d4ec3543bd45a228383ff9a0aa1ccbedc8c4: Status 404 returned error can't find the container with id f0a6de2a4b1a3adf8cdd3bdafc40d4ec3543bd45a228383ff9a0aa1ccbedc8c4
Sep 06 19:20:35.180814 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:35.180794    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 19:20:35.182241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:35.182217    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:35.537450 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:35.537401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc" event={"ID":"8bfbac4f-4890-4bdb-9f21-72e6e476d78a","Type":"ContainerStarted","Data":"f0a6de2a4b1a3adf8cdd3bdafc40d4ec3543bd45a228383ff9a0aa1ccbedc8c4"}
Sep 06 19:20:37.596954 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:37.596922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:37.713873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:37.713832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:37.798905 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:37.798871    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:38.323597 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:38.323561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:38.423050 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:38.423020    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:38.563400 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:38.563361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc" event={"ID":"8bfbac4f-4890-4bdb-9f21-72e6e476d78a","Type":"ContainerStarted","Data":"a7eec15d11d8d43b9570e2153cb1e895f1fa754251d66bd4ffb1a7fdec75ddbd"}
Sep 06 19:20:38.961230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:38.961194    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:39.074176 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:39.074148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:39.570134 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:39.570037    2581 generic.go:356] "Generic (PLEG): container finished" podID="8bfbac4f-4890-4bdb-9f21-72e6e476d78a" containerID="a7eec15d11d8d43b9570e2153cb1e895f1fa754251d66bd4ffb1a7fdec75ddbd" exitCode=0
Sep 06 19:20:39.570134 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:39.570092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc" event={"ID":"8bfbac4f-4890-4bdb-9f21-72e6e476d78a","Type":"ContainerDied","Data":"a7eec15d11d8d43b9570e2153cb1e895f1fa754251d66bd4ffb1a7fdec75ddbd"}
Sep 06 19:20:40.578526 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:40.576514    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc" event={"ID":"8bfbac4f-4890-4bdb-9f21-72e6e476d78a","Type":"ContainerStarted","Data":"1a0c0101ab92142f014f618e200506cfefe042f51e0564c92ffd765e5d34bc47"}
Sep 06 19:20:40.578526 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:40.577391    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:40.598570 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:40.598502    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc" podStartSLOduration=1.894664914 podStartE2EDuration="6.59848056s" totalImagesPullingTime="3.582629336s" totalInitContainerRuntime="1.12118631s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:20:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:20:40.597237543 +0000 UTC m=+568.877690349" watchObservedRunningTime="2026-09-06 19:20:40.59848056 +0000 UTC m=+568.878933355"
Sep 06 19:20:46.398947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:46.398913    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:46.500515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:46.500484    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:47.415351 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:47.415311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:47.515975 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:47.515936    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:47.846534 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:47.846456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:47.947116 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:47.947076    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:48.612967 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:48.612927    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:48.714632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:48.714604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:48.814443 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:48.814409    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:52.588775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:52.588747    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d4b9b44b9-x9qtc"
Sep 06 19:20:53.999178 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:53.999147    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:54.094384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:54.094345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:54.403020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:54.402990    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:54.651922 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:54.651880    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:55.681237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:55.681200    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:55.744699 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:55.744661    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:20:56.559842 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:20:56.559812    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:00.913353 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:00.913318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:01.014372 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.014333    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:01.067387 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.067354    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"]
Sep 06 19:21:01.082433 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.082401    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.082892 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.082860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"]
Sep 06 19:21:01.084597 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.084575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rm45x\""
Sep 06 19:21:01.084736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.084584    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 06 19:21:01.106840 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.106814    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:01.147152 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.147097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.147339 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.147217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.147339 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.147310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.147448 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.147344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjctc\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-kube-api-access-xjctc\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.147448 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.147383    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.147448 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.147422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.147559 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.147453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.147559 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.147470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.147559 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.147488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.244641 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.244553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:01.248742 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.248718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.248837 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.248769    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.248837 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.248802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjctc\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-kube-api-access-xjctc\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.248837 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.248826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.248966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.248844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.248966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.248861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.248966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.248875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.249124 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.249028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.249124 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.249093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.249234 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.249183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.249288 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.249258    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.249384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.249360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.249432 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.249392    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.250043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.250021    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.251534 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.251512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.251661 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.251513    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.257210 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.257188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjctc\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-kube-api-access-xjctc\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.257306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.257289    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcgt58v\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.260907 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.260882    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"]
Sep 06 19:21:01.266395 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.266365    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.275643 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.275617    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"]
Sep 06 19:21:01.350212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.350170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.350413 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.350240    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.350413 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.350270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.350413 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.350297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.350413 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.350359    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.350586 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.350474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.350586 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.350509    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.350649 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.350614    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.350679 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.350663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgfhv\" (UniqueName: \"kubernetes.io/projected/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-kube-api-access-xgfhv\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.393600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.393557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:01.440639 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.440603    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:01.451620 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.451590    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.451750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.451635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.451750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.451667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.451750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.451701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.451907 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.451761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.451907 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.451785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.451907 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.451833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.451907 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.451877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgfhv\" (UniqueName: \"kubernetes.io/projected/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-kube-api-access-xgfhv\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.452136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.451908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.452586 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.452527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.452788 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.452766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.452890 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.452863    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.453052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.453012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.453477 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.453457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.455273 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.455232    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.455943 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.455914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.463606 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.463544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.464849 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.464819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgfhv\" (UniqueName: \"kubernetes.io/projected/44c88b6f-e60d-45ff-a83c-468ee68f4ba1-kube-api-access-xgfhv\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h\" (UID: \"44c88b6f-e60d-45ff-a83c-468ee68f4ba1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.581841 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.581749    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:01.671043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.671008    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"]
Sep 06 19:21:01.674019 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:21:01.673983    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a4c7c6_e8db_44bb_9849_f93abfd51ed2.slice/crio-4b2d46cc1c66adfb0e7eb3467aef2e350d02ed787ebd9e89a12fd2d0e31bc9e3 WatchSource:0}: Error finding container 4b2d46cc1c66adfb0e7eb3467aef2e350d02ed787ebd9e89a12fd2d0e31bc9e3: Status 404 returned error can't find the container with id 4b2d46cc1c66adfb0e7eb3467aef2e350d02ed787ebd9e89a12fd2d0e31bc9e3
Sep 06 19:21:01.852569 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:01.852540    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"]
Sep 06 19:21:01.855074 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:21:01.855027    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c88b6f_e60d_45ff_a83c_468ee68f4ba1.slice/crio-ab2efe548520c224a23b6f8c976355e3afdeb879ea232e4716bbc8a6c8a26e2c WatchSource:0}: Error finding container ab2efe548520c224a23b6f8c976355e3afdeb879ea232e4716bbc8a6c8a26e2c: Status 404 returned error can't find the container with id ab2efe548520c224a23b6f8c976355e3afdeb879ea232e4716bbc8a6c8a26e2c
Sep 06 19:21:02.679538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:02.679489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h" event={"ID":"44c88b6f-e60d-45ff-a83c-468ee68f4ba1","Type":"ContainerStarted","Data":"ab2efe548520c224a23b6f8c976355e3afdeb879ea232e4716bbc8a6c8a26e2c"}
Sep 06 19:21:02.681084 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:02.681031    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v" event={"ID":"55a4c7c6-e8db-44bb-9849-f93abfd51ed2","Type":"ContainerStarted","Data":"4b2d46cc1c66adfb0e7eb3467aef2e350d02ed787ebd9e89a12fd2d0e31bc9e3"}
Sep 06 19:21:04.465711 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:04.465664    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:21:04.465996 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:04.465754    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:21:04.465996 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:04.465790    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:21:04.691966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:04.691925    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h" event={"ID":"44c88b6f-e60d-45ff-a83c-468ee68f4ba1","Type":"ContainerStarted","Data":"0d22751cecb8e6b4becd09d3fd6d4bd9a719ec8d1ee223f03930cb1b14d3cd7d"}
Sep 06 19:21:04.709310 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:04.709247    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h" podStartSLOduration=1.10094671 podStartE2EDuration="3.709222803s" totalImagesPullingTime="2.608276093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:21:04.707659976 +0000 UTC m=+592.988112783" watchObservedRunningTime="2026-09-06 19:21:04.709222803 +0000 UTC m=+592.989675599"
Sep 06 19:21:04.968884 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:04.968829    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:05.069115 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:05.069054    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:05.211302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:05.211265    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:05.280340 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:05.280258    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:05.582956 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:05.582863    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:05.587729 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:05.587702    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:05.696020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:05.695991    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:05.696885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:05.696862    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h"
Sep 06 19:21:05.744750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:05.744714    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"]
Sep 06 19:21:05.957957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:05.957917    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:10.119375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:10.119342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:10.167720 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:10.167686    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:12.205943 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.205913    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:21:12.206399 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.206185    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:21:12.211282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.211257    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:21:12.211426 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.211262    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:21:12.472930 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.472846    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"]
Sep 06 19:21:12.477919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.477894    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.480331 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.480306    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 06 19:21:12.480747 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.480723    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ldlmh\""
Sep 06 19:21:12.480747 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.480740    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 06 19:21:12.480907 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.480851    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 06 19:21:12.480907 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.480875    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 06 19:21:12.480988 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.480942    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 06 19:21:12.489010 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.488980    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"]
Sep 06 19:21:12.581042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.581006    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:12.657951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.657917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-manager-config\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.657951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.657955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-metrics-cert\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.658216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.657981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mc96\" (UniqueName: \"kubernetes.io/projected/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-kube-api-access-9mc96\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.658216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.658125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-cert\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.681443 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.681408    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:12.758676 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.758576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mc96\" (UniqueName: \"kubernetes.io/projected/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-kube-api-access-9mc96\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.758676 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.758663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-cert\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.758904 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.758742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-manager-config\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.758904 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.758767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-metrics-cert\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.759494 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.759473    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-manager-config\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.761600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.761577    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-cert\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.761735 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.761717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-metrics-cert\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.767047 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.767011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mc96\" (UniqueName: \"kubernetes.io/projected/b56c21ec-2ef5-427e-be6f-d00c51d9a71c-kube-api-access-9mc96\") pod \"lws-controller-manager-5c4c9c58b7-g5l2j\" (UID: \"b56c21ec-2ef5-427e-be6f-d00c51d9a71c\") " pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:12.789005 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:12.788963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:13.061785 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:13.061703    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"]
Sep 06 19:21:13.065327 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:21:13.065298    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb56c21ec_2ef5_427e_be6f_d00c51d9a71c.slice/crio-c2e5802803043591c4e9e82c3b454d46a2f62e5d8593417f48afd8748df923a8 WatchSource:0}: Error finding container c2e5802803043591c4e9e82c3b454d46a2f62e5d8593417f48afd8748df923a8: Status 404 returned error can't find the container with id c2e5802803043591c4e9e82c3b454d46a2f62e5d8593417f48afd8748df923a8
Sep 06 19:21:13.732950 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:13.732916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j" event={"ID":"b56c21ec-2ef5-427e-be6f-d00c51d9a71c","Type":"ContainerStarted","Data":"c2e5802803043591c4e9e82c3b454d46a2f62e5d8593417f48afd8748df923a8"}
Sep 06 19:21:14.595420 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.595289    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:14.638001 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.637955    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nstws"]
Sep 06 19:21:14.641568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.641540    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:21:14.643822 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.643799    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 06 19:21:14.643975 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.643823    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rr8cn\""
Sep 06 19:21:14.654295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.654262    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nstws"]
Sep 06 19:21:14.656998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.656657    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"]
Sep 06 19:21:14.660833 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.660809    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:14.663054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.662857    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 06 19:21:14.663209 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.663124    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-swnmt\""
Sep 06 19:21:14.667351 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.666279    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"]
Sep 06 19:21:14.676865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.676813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:14.777426 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.777389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwlnq\" (UniqueName: \"kubernetes.io/projected/716ad3b7-1459-48be-b8db-b281a92f213f-kube-api-access-fwlnq\") pod \"kserve-controller-manager-d6f4b65b8-nstws\" (UID: \"716ad3b7-1459-48be-b8db-b281a92f213f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:21:14.777859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.777488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5n92\" (UniqueName: \"kubernetes.io/projected/831b3d1a-6948-40b5-88a6-1fe9a71d6901-kube-api-access-g5n92\") pod \"llmisvc-controller-manager-777c6bd7d5-xhwjk\" (UID: \"831b3d1a-6948-40b5-88a6-1fe9a71d6901\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:14.777859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.777536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/716ad3b7-1459-48be-b8db-b281a92f213f-cert\") pod \"kserve-controller-manager-d6f4b65b8-nstws\" (UID: \"716ad3b7-1459-48be-b8db-b281a92f213f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:21:14.777859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.777615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/831b3d1a-6948-40b5-88a6-1fe9a71d6901-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xhwjk\" (UID: \"831b3d1a-6948-40b5-88a6-1fe9a71d6901\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:14.878836 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.878750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/831b3d1a-6948-40b5-88a6-1fe9a71d6901-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xhwjk\" (UID: \"831b3d1a-6948-40b5-88a6-1fe9a71d6901\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:14.878836 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.878820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwlnq\" (UniqueName: \"kubernetes.io/projected/716ad3b7-1459-48be-b8db-b281a92f213f-kube-api-access-fwlnq\") pod \"kserve-controller-manager-d6f4b65b8-nstws\" (UID: \"716ad3b7-1459-48be-b8db-b281a92f213f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:21:14.879081 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.878884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5n92\" (UniqueName: \"kubernetes.io/projected/831b3d1a-6948-40b5-88a6-1fe9a71d6901-kube-api-access-g5n92\") pod \"llmisvc-controller-manager-777c6bd7d5-xhwjk\" (UID: \"831b3d1a-6948-40b5-88a6-1fe9a71d6901\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:14.879081 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.878924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/716ad3b7-1459-48be-b8db-b281a92f213f-cert\") pod \"kserve-controller-manager-d6f4b65b8-nstws\" (UID: \"716ad3b7-1459-48be-b8db-b281a92f213f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:21:14.881898 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.881865    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/716ad3b7-1459-48be-b8db-b281a92f213f-cert\") pod \"kserve-controller-manager-d6f4b65b8-nstws\" (UID: \"716ad3b7-1459-48be-b8db-b281a92f213f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:21:14.882110 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.881865    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/831b3d1a-6948-40b5-88a6-1fe9a71d6901-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xhwjk\" (UID: \"831b3d1a-6948-40b5-88a6-1fe9a71d6901\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:14.886837 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.886809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:14.893613 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.893585    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwlnq\" (UniqueName: \"kubernetes.io/projected/716ad3b7-1459-48be-b8db-b281a92f213f-kube-api-access-fwlnq\") pod \"kserve-controller-manager-d6f4b65b8-nstws\" (UID: \"716ad3b7-1459-48be-b8db-b281a92f213f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:21:14.893755 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.893668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5n92\" (UniqueName: \"kubernetes.io/projected/831b3d1a-6948-40b5-88a6-1fe9a71d6901-kube-api-access-g5n92\") pod \"llmisvc-controller-manager-777c6bd7d5-xhwjk\" (UID: \"831b3d1a-6948-40b5-88a6-1fe9a71d6901\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:14.957706 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.957670    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:21:14.974604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:14.974566    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:15.024053 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:15.024016    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:15.844027 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:15.843949    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:15.943237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:15.943207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:16.089160 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:16.089087    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:16.103469 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:16.103443    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"]
Sep 06 19:21:16.106102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:16.105319    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nstws"]
Sep 06 19:21:16.748653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:16.748617    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws" event={"ID":"716ad3b7-1459-48be-b8db-b281a92f213f","Type":"ContainerStarted","Data":"c990961c488b0f1a9cfac96896d5edcd891e214b562e3537bc2f8bf498b355a3"}
Sep 06 19:21:16.750396 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:16.750358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j" event={"ID":"b56c21ec-2ef5-427e-be6f-d00c51d9a71c","Type":"ContainerStarted","Data":"ca08cbdcc8dbabd09a61a89cbc2d89a99eb324ac6344aaffd96e98f2e48b82a7"}
Sep 06 19:21:16.750528 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:16.750424    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:16.751624 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:16.751602    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk" event={"ID":"831b3d1a-6948-40b5-88a6-1fe9a71d6901","Type":"ContainerStarted","Data":"265e87f094f07237471d83bed7b19f5f5b9fb42d4800405bf264b07a190acc36"}
Sep 06 19:21:16.766232 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:16.766182    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j" podStartSLOduration=1.928043979 podStartE2EDuration="4.766164026s" totalImagesPullingTime="2.838120047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:21:16.764250671 +0000 UTC m=+605.044703464" watchObservedRunningTime="2026-09-06 19:21:16.766164026 +0000 UTC m=+605.046616819"
Sep 06 19:21:20.106194 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.106158    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:21:20.106526 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.106233    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:21:20.106526 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.106326    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:21:20.775905 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.775865    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws" event={"ID":"716ad3b7-1459-48be-b8db-b281a92f213f","Type":"ContainerStarted","Data":"6a54091a7788afbddf84e55898a1cd71f96ae5f793056ddf10dab1b30b9cc258"}
Sep 06 19:21:20.776125 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.776042    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:21:20.777565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.777535    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v" event={"ID":"55a4c7c6-e8db-44bb-9849-f93abfd51ed2","Type":"ContainerStarted","Data":"b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0"}
Sep 06 19:21:20.777688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.777583    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v" podUID="55a4c7c6-e8db-44bb-9849-f93abfd51ed2" containerName="istio-proxy" containerID="cri-o://b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0" gracePeriod=30
Sep 06 19:21:20.779282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.779230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk" event={"ID":"831b3d1a-6948-40b5-88a6-1fe9a71d6901","Type":"ContainerStarted","Data":"227bf4390dce0520be210211480243b03efaadb574f2a9d6440d3b53b92dc93a"}
Sep 06 19:21:20.779390 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.779292    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:20.792782 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.792723    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws" podStartSLOduration=3.174366788 podStartE2EDuration="6.792704859s" totalImagesPullingTime="3.618338071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:21:20.790868963 +0000 UTC m=+609.071321758" watchObservedRunningTime="2026-09-06 19:21:20.792704859 +0000 UTC m=+609.073157653"
Sep 06 19:21:20.805228 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.805170    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk" podStartSLOduration=3.186834987 podStartE2EDuration="6.80515476s" totalImagesPullingTime="3.618319773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:21:20.804266191 +0000 UTC m=+609.084719014" watchObservedRunningTime="2026-09-06 19:21:20.80515476 +0000 UTC m=+609.085607554"
Sep 06 19:21:20.823196 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:20.823130    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v" podStartSLOduration=1.393195416 podStartE2EDuration="19.823104438s" totalImagesPullingTime="18.429909022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:21:20.820481542 +0000 UTC m=+609.100934359" watchObservedRunningTime="2026-09-06 19:21:20.823104438 +0000 UTC m=+609.103557234"
Sep 06 19:21:26.039385 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.039287    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:26.090588 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090556    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-socket\") pod \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") "
Sep 06 19:21:26.090779 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090597    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-podinfo\") pod \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") "
Sep 06 19:21:26.090779 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090621    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-token\") pod \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") "
Sep 06 19:21:26.090779 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-credential-socket\") pod \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") "
Sep 06 19:21:26.090779 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090660    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-data\") pod \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") "
Sep 06 19:21:26.090779 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090697    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-certs\") pod \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") "
Sep 06 19:21:26.090779 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090741    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istiod-ca-cert\") pod \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") "
Sep 06 19:21:26.090779 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090764    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjctc\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-kube-api-access-xjctc\") pod \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") "
Sep 06 19:21:26.091238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090810    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-envoy\") pod \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\" (UID: \"55a4c7c6-e8db-44bb-9849-f93abfd51ed2\") "
Sep 06 19:21:26.091238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.090992    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "55a4c7c6-e8db-44bb-9849-f93abfd51ed2" (UID: "55a4c7c6-e8db-44bb-9849-f93abfd51ed2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:21:26.091238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.091032    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-data" (OuterVolumeSpecName: "istio-data") pod "55a4c7c6-e8db-44bb-9849-f93abfd51ed2" (UID: "55a4c7c6-e8db-44bb-9849-f93abfd51ed2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:21:26.091238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.091049    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "55a4c7c6-e8db-44bb-9849-f93abfd51ed2" (UID: "55a4c7c6-e8db-44bb-9849-f93abfd51ed2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:21:26.091238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.091216    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:21:26.091639 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.091266    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:21:26.091639 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.091268    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "55a4c7c6-e8db-44bb-9849-f93abfd51ed2" (UID: "55a4c7c6-e8db-44bb-9849-f93abfd51ed2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:21:26.091639 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.091284    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:21:26.091639 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.091341    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "55a4c7c6-e8db-44bb-9849-f93abfd51ed2" (UID: "55a4c7c6-e8db-44bb-9849-f93abfd51ed2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:21:26.093110 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.093058    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-token" (OuterVolumeSpecName: "istio-token") pod "55a4c7c6-e8db-44bb-9849-f93abfd51ed2" (UID: "55a4c7c6-e8db-44bb-9849-f93abfd51ed2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:21:26.093110 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.093101    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "55a4c7c6-e8db-44bb-9849-f93abfd51ed2" (UID: "55a4c7c6-e8db-44bb-9849-f93abfd51ed2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:21:26.093545 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.093523    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "55a4c7c6-e8db-44bb-9849-f93abfd51ed2" (UID: "55a4c7c6-e8db-44bb-9849-f93abfd51ed2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:21:26.093585 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.093543    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-kube-api-access-xjctc" (OuterVolumeSpecName: "kube-api-access-xjctc") pod "55a4c7c6-e8db-44bb-9849-f93abfd51ed2" (UID: "55a4c7c6-e8db-44bb-9849-f93abfd51ed2"). InnerVolumeSpecName "kube-api-access-xjctc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:21:26.192583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.192542    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:21:26.192583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.192576    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:21:26.192583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.192586    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:21:26.192583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.192598    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:21:26.192839 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.192606    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjctc\" (UniqueName: \"kubernetes.io/projected/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-kube-api-access-xjctc\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:21:26.192839 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.192614    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a4c7c6-e8db-44bb-9849-f93abfd51ed2-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:21:26.808326 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.808289    2581 generic.go:356] "Generic (PLEG): container finished" podID="55a4c7c6-e8db-44bb-9849-f93abfd51ed2" containerID="b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0" exitCode=0
Sep 06 19:21:26.808485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.808341    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"
Sep 06 19:21:26.808485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.808376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v" event={"ID":"55a4c7c6-e8db-44bb-9849-f93abfd51ed2","Type":"ContainerDied","Data":"b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0"}
Sep 06 19:21:26.808485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.808431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v" event={"ID":"55a4c7c6-e8db-44bb-9849-f93abfd51ed2","Type":"ContainerDied","Data":"4b2d46cc1c66adfb0e7eb3467aef2e350d02ed787ebd9e89a12fd2d0e31bc9e3"}
Sep 06 19:21:26.808485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.808460    2581 scope.go:117] "RemoveContainer" containerID="b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0"
Sep 06 19:21:26.817578 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.817561    2581 scope.go:117] "RemoveContainer" containerID="b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0"
Sep 06 19:21:26.817911 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:21:26.817888    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0\": container with ID starting with b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0 not found: ID does not exist" containerID="b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0"
Sep 06 19:21:26.817978 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.817918    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0"} err="failed to get container status \"b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0\": rpc error: code = NotFound desc = could not find container \"b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0\": container with ID starting with b392982b00246f00645b143fc09c239188b9e148f3ee0e0768b2a638082165c0 not found: ID does not exist"
Sep 06 19:21:26.824369 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.824341    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"]
Sep 06 19:21:26.830113 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:26.830088    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcgt58v"]
Sep 06 19:21:27.760950 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:27.760923    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c4c9c58b7-g5l2j"
Sep 06 19:21:28.265514 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:28.265473    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55a4c7c6-e8db-44bb-9849-f93abfd51ed2" path="/var/lib/kubelet/pods/55a4c7c6-e8db-44bb-9849-f93abfd51ed2/volumes"
Sep 06 19:21:29.578259 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:29.578223    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:29.679497 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:29.679464    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:48.143516 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:48.143482    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:49.056871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:49.056836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:49.102376 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:49.102347    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:21:51.787409 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:51.787380    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xhwjk"
Sep 06 19:21:51.790383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:21:51.790362    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nstws"
Sep 06 19:22:06.106940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:06.106901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:06.172335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:06.172300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:18.383711 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:18.383675    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:20.426910 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.426869    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"]
Sep 06 19:22:20.427309 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.427268    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55a4c7c6-e8db-44bb-9849-f93abfd51ed2" containerName="istio-proxy"
Sep 06 19:22:20.427309 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.427281    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55a4c7c6-e8db-44bb-9849-f93abfd51ed2" containerName="istio-proxy"
Sep 06 19:22:20.427397 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.427356    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55a4c7c6-e8db-44bb-9849-f93abfd51ed2" containerName="istio-proxy"
Sep 06 19:22:20.431676 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.431655    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"
Sep 06 19:22:20.433986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.433961    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 06 19:22:20.433986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.433974    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qv6cm\""
Sep 06 19:22:20.434235 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.434057    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 06 19:22:20.434575 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.434552    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 06 19:22:20.441049 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.441021    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"]
Sep 06 19:22:20.472797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.472755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv9cb\" (UniqueName: \"kubernetes.io/projected/ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1-kube-api-access-fv9cb\") pod \"dns-operator-controller-manager-5985564776-5lfzh\" (UID: \"ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"
Sep 06 19:22:20.573773 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.573728    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv9cb\" (UniqueName: \"kubernetes.io/projected/ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1-kube-api-access-fv9cb\") pod \"dns-operator-controller-manager-5985564776-5lfzh\" (UID: \"ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"
Sep 06 19:22:20.587343 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.587314    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv9cb\" (UniqueName: \"kubernetes.io/projected/ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1-kube-api-access-fv9cb\") pod \"dns-operator-controller-manager-5985564776-5lfzh\" (UID: \"ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"
Sep 06 19:22:20.743360 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.743255    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"
Sep 06 19:22:20.980284 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:20.980251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:21.017357 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:21.017274    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"]
Sep 06 19:22:21.021001 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:22:21.020972    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4c9d43_5ac7_4f9e_b671_bd6cb44e8bf1.slice/crio-7bed26b0502f118320982f03d4d879f6d6ab27ed66d96a97ed4b0a42472fa0fe WatchSource:0}: Error finding container 7bed26b0502f118320982f03d4d879f6d6ab27ed66d96a97ed4b0a42472fa0fe: Status 404 returned error can't find the container with id 7bed26b0502f118320982f03d4d879f6d6ab27ed66d96a97ed4b0a42472fa0fe
Sep 06 19:22:21.061235 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:21.061204    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh" event={"ID":"ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1","Type":"ContainerStarted","Data":"7bed26b0502f118320982f03d4d879f6d6ab27ed66d96a97ed4b0a42472fa0fe"}
Sep 06 19:22:21.381636 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:21.381531    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:21.781353 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:21.781272    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:23.214178 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:23.214144    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:24.079366 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:24.079331    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh" event={"ID":"ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1","Type":"ContainerStarted","Data":"75667a52202b4c22af77f45bef7e4fa9b5d7f5aafcdd1c374b42186829b93163"}
Sep 06 19:22:24.079534 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:24.079492    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"
Sep 06 19:22:24.095525 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:24.095468    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh" podStartSLOduration=1.194340887 podStartE2EDuration="4.095449593s" totalImagesPullingTime="2.901108706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:22:24.093695143 +0000 UTC m=+672.374147937" watchObservedRunningTime="2026-09-06 19:22:24.095449593 +0000 UTC m=+672.375902386"
Sep 06 19:22:25.050839 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:25.050801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:25.155783 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:25.155751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:25.250650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:25.250598    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:26.626251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:26.626218    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"]
Sep 06 19:22:26.629806 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:26.629782    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:26.632114 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:26.632093    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-dhxlr\""
Sep 06 19:22:26.639210 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:26.639178    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"]
Sep 06 19:22:26.730591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:26.730547    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lszz7\" (UniqueName: \"kubernetes.io/projected/abb24fe7-ab6b-4c77-b68a-cb471638f7a0-kube-api-access-lszz7\") pod \"limitador-operator-controller-manager-6c549b6d85-b299f\" (UID: \"abb24fe7-ab6b-4c77-b68a-cb471638f7a0\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:26.784608 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:26.784572    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:26.831417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:26.831375    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lszz7\" (UniqueName: \"kubernetes.io/projected/abb24fe7-ab6b-4c77-b68a-cb471638f7a0-kube-api-access-lszz7\") pod \"limitador-operator-controller-manager-6c549b6d85-b299f\" (UID: \"abb24fe7-ab6b-4c77-b68a-cb471638f7a0\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:26.840269 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:26.840243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lszz7\" (UniqueName: \"kubernetes.io/projected/abb24fe7-ab6b-4c77-b68a-cb471638f7a0-kube-api-access-lszz7\") pod \"limitador-operator-controller-manager-6c549b6d85-b299f\" (UID: \"abb24fe7-ab6b-4c77-b68a-cb471638f7a0\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:26.940989 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:26.940954    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:27.183424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:27.183393    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:27.217413 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:27.217383    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"]
Sep 06 19:22:27.219507 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:22:27.219474    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabb24fe7_ab6b_4c77_b68a_cb471638f7a0.slice/crio-792e0d562ea66899452013488d83c8ac9af5642f2ccb8cd22d238d089fbbad9e WatchSource:0}: Error finding container 792e0d562ea66899452013488d83c8ac9af5642f2ccb8cd22d238d089fbbad9e: Status 404 returned error can't find the container with id 792e0d562ea66899452013488d83c8ac9af5642f2ccb8cd22d238d089fbbad9e
Sep 06 19:22:27.819328 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:27.819291    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:28.099108 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:28.099009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f" event={"ID":"abb24fe7-ab6b-4c77-b68a-cb471638f7a0","Type":"ContainerStarted","Data":"792e0d562ea66899452013488d83c8ac9af5642f2ccb8cd22d238d089fbbad9e"}
Sep 06 19:22:29.105853 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.105749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f" event={"ID":"abb24fe7-ab6b-4c77-b68a-cb471638f7a0","Type":"ContainerStarted","Data":"971b662d9dd67e076ab38cf369735aedfc3a027a8cd0d8d7a4d1d288bd32bf4b"}
Sep 06 19:22:29.106289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.105936    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:29.124598 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.124543    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f" podStartSLOduration=1.506424643 podStartE2EDuration="3.124527242s" totalImagesPullingTime="1.618102599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:22:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:22:29.12323678 +0000 UTC m=+677.403689575" watchObservedRunningTime="2026-09-06 19:22:29.124527242 +0000 UTC m=+677.404980036"
Sep 06 19:22:29.222207 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.222167    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-r79vb"]
Sep 06 19:22:29.225650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.225628    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-r79vb"
Sep 06 19:22:29.227953 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.227931    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8xdnp\""
Sep 06 19:22:29.236833 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.236802    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-r79vb"]
Sep 06 19:22:29.352389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.352348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzcpx\" (UniqueName: \"kubernetes.io/projected/4aa210f1-45f2-4319-8335-db666ca52357-kube-api-access-qzcpx\") pod \"authorino-operator-68d88948d8-r79vb\" (UID: \"4aa210f1-45f2-4319-8335-db666ca52357\") " pod="kuadrant-system/authorino-operator-68d88948d8-r79vb"
Sep 06 19:22:29.453531 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.453492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzcpx\" (UniqueName: \"kubernetes.io/projected/4aa210f1-45f2-4319-8335-db666ca52357-kube-api-access-qzcpx\") pod \"authorino-operator-68d88948d8-r79vb\" (UID: \"4aa210f1-45f2-4319-8335-db666ca52357\") " pod="kuadrant-system/authorino-operator-68d88948d8-r79vb"
Sep 06 19:22:29.461484 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.461457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzcpx\" (UniqueName: \"kubernetes.io/projected/4aa210f1-45f2-4319-8335-db666ca52357-kube-api-access-qzcpx\") pod \"authorino-operator-68d88948d8-r79vb\" (UID: \"4aa210f1-45f2-4319-8335-db666ca52357\") " pod="kuadrant-system/authorino-operator-68d88948d8-r79vb"
Sep 06 19:22:29.537555 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.537517    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-r79vb"
Sep 06 19:22:29.809931 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:29.809899    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-r79vb"]
Sep 06 19:22:29.812160 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:22:29.812132    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa210f1_45f2_4319_8335_db666ca52357.slice/crio-34d74d4f6e9318080b5c3aaee4dbf7be201eef95b62da1420d37cbe46ab6d8b6 WatchSource:0}: Error finding container 34d74d4f6e9318080b5c3aaee4dbf7be201eef95b62da1420d37cbe46ab6d8b6: Status 404 returned error can't find the container with id 34d74d4f6e9318080b5c3aaee4dbf7be201eef95b62da1420d37cbe46ab6d8b6
Sep 06 19:22:30.021770 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:30.021739    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:30.111986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:30.111892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-r79vb" event={"ID":"4aa210f1-45f2-4319-8335-db666ca52357","Type":"ContainerStarted","Data":"34d74d4f6e9318080b5c3aaee4dbf7be201eef95b62da1420d37cbe46ab6d8b6"}
Sep 06 19:22:30.798528 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:30.798489    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:30.899153 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:30.899113    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:31.633830 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:31.633801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:31.856542 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:31.856509    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:32.125122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:32.125019    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-r79vb" event={"ID":"4aa210f1-45f2-4319-8335-db666ca52357","Type":"ContainerStarted","Data":"07a942dd14ba8a93731994ccc49a88dae2a9dbe68bb9dcdf02b325380d4102f4"}
Sep 06 19:22:32.125268 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:32.125176    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-r79vb"
Sep 06 19:22:32.143038 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:32.142982    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-r79vb" podStartSLOduration=1.876208243 podStartE2EDuration="3.142963189s" totalImagesPullingTime="1.266754946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:22:32.141694487 +0000 UTC m=+680.422147282" watchObservedRunningTime="2026-09-06 19:22:32.142963189 +0000 UTC m=+680.423415983"
Sep 06 19:22:35.086869 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:35.086831    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-5lfzh"
Sep 06 19:22:35.655522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:35.655487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:35.751393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:35.751357    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:35.992934 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:35.992850    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:36.093626 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:36.093592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:36.388442 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:36.388411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:36.485118 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:36.485081    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:40.114404 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:40.114361    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:42.400998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.400962    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:42.488308 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.488272    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:42.948754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.948720    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"]
Sep 06 19:22:42.948934 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.948911    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" containerName="manager" containerID="cri-o://971b662d9dd67e076ab38cf369735aedfc3a027a8cd0d8d7a4d1d288bd32bf4b" gracePeriod=2
Sep 06 19:22:42.960431 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.959459    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"]
Sep 06 19:22:42.968763 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:22:42.968722    2581 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"abb24fe7-ab6b-4c77-b68a-cb471638f7a0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-b299f\": pods \"limitador-operator-controller-manager-6c549b6d85-b299f\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:42.979493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.979459    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"]
Sep 06 19:22:42.979869 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.979856    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" containerName="manager"
Sep 06 19:22:42.979919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.979871    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" containerName="manager"
Sep 06 19:22:42.979954 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.979947    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" containerName="manager"
Sep 06 19:22:42.983147 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.983124    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"
Sep 06 19:22:42.993660 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:42.993625    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"]
Sep 06 19:22:43.012724 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:22:43.012681    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-b299f\" is forbidden: User \"system:node:ip-10-0-135-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-200.ec2.internal' and this object" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:43.075021 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.074981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rkp5\" (UniqueName: \"kubernetes.io/projected/3f2a11d7-c9b3-4c22-9986-5ae7682a5a60-kube-api-access-2rkp5\") pod \"limitador-operator-controller-manager-6c549b6d85-knvzq\" (UID: \"3f2a11d7-c9b3-4c22-9986-5ae7682a5a60\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"
Sep 06 19:22:43.133242 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.133209    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-r79vb"
Sep 06 19:22:43.150970 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:22:43.150924    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-b299f\" is forbidden: User \"system:node:ip-10-0-135-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-200.ec2.internal' and this object" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:43.175789 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.175747    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rkp5\" (UniqueName: \"kubernetes.io/projected/3f2a11d7-c9b3-4c22-9986-5ae7682a5a60-kube-api-access-2rkp5\") pod \"limitador-operator-controller-manager-6c549b6d85-knvzq\" (UID: \"3f2a11d7-c9b3-4c22-9986-5ae7682a5a60\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"
Sep 06 19:22:43.181561 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.181530    2581 generic.go:356] "Generic (PLEG): container finished" podID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" containerID="971b662d9dd67e076ab38cf369735aedfc3a027a8cd0d8d7a4d1d288bd32bf4b" exitCode=0
Sep 06 19:22:43.184161 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.184121    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rkp5\" (UniqueName: \"kubernetes.io/projected/3f2a11d7-c9b3-4c22-9986-5ae7682a5a60-kube-api-access-2rkp5\") pod \"limitador-operator-controller-manager-6c549b6d85-knvzq\" (UID: \"3f2a11d7-c9b3-4c22-9986-5ae7682a5a60\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"
Sep 06 19:22:43.207418 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.207340    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:43.209298 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:22:43.209266    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-b299f\" is forbidden: User \"system:node:ip-10-0-135-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-200.ec2.internal' and this object" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:43.337261 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.337223    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"
Sep 06 19:22:43.378324 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.378289    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lszz7\" (UniqueName: \"kubernetes.io/projected/abb24fe7-ab6b-4c77-b68a-cb471638f7a0-kube-api-access-lszz7\") pod \"abb24fe7-ab6b-4c77-b68a-cb471638f7a0\" (UID: \"abb24fe7-ab6b-4c77-b68a-cb471638f7a0\") "
Sep 06 19:22:43.381273 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.381246    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb24fe7-ab6b-4c77-b68a-cb471638f7a0-kube-api-access-lszz7" (OuterVolumeSpecName: "kube-api-access-lszz7") pod "abb24fe7-ab6b-4c77-b68a-cb471638f7a0" (UID: "abb24fe7-ab6b-4c77-b68a-cb471638f7a0"). InnerVolumeSpecName "kube-api-access-lszz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:22:43.481123 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.481003    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lszz7\" (UniqueName: \"kubernetes.io/projected/abb24fe7-ab6b-4c77-b68a-cb471638f7a0-kube-api-access-lszz7\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:22:43.643621 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:43.643590    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"]
Sep 06 19:22:43.645634 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:22:43.645608    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f2a11d7_c9b3_4c22_9986_5ae7682a5a60.slice/crio-a19ec6ba914b67dec705b1d09c724b8fb1906e85f84ea817d26920c75047a67e WatchSource:0}: Error finding container a19ec6ba914b67dec705b1d09c724b8fb1906e85f84ea817d26920c75047a67e: Status 404 returned error can't find the container with id a19ec6ba914b67dec705b1d09c724b8fb1906e85f84ea817d26920c75047a67e
Sep 06 19:22:44.189374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:44.189337    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq" event={"ID":"3f2a11d7-c9b3-4c22-9986-5ae7682a5a60","Type":"ContainerStarted","Data":"0c7ea652187d6820fec4c296a5a7a9970c8bc6b61b850538cb3be4b418736075"}
Sep 06 19:22:44.189585 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:44.189397    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"
Sep 06 19:22:44.189585 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:44.189419    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq" event={"ID":"3f2a11d7-c9b3-4c22-9986-5ae7682a5a60","Type":"ContainerStarted","Data":"a19ec6ba914b67dec705b1d09c724b8fb1906e85f84ea817d26920c75047a67e"}
Sep 06 19:22:44.190892 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:44.190872    2581 scope.go:117] "RemoveContainer" containerID="971b662d9dd67e076ab38cf369735aedfc3a027a8cd0d8d7a4d1d288bd32bf4b"
Sep 06 19:22:44.190983 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:44.190880    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:44.191455 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:22:44.191429    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-b299f\" is forbidden: User \"system:node:ip-10-0-135-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-200.ec2.internal' and this object" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:44.216663 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:22:44.216566    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-b299f\" is forbidden: User \"system:node:ip-10-0-135-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-200.ec2.internal' and this object" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b299f"
Sep 06 19:22:44.217152 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:44.217113    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq" podStartSLOduration=2.217096817 podStartE2EDuration="2.217096817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:22:44.214929296 +0000 UTC m=+692.495382091" watchObservedRunningTime="2026-09-06 19:22:44.217096817 +0000 UTC m=+692.497549610"
Sep 06 19:22:44.265861 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:44.265825    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abb24fe7-ab6b-4c77-b68a-cb471638f7a0" path="/var/lib/kubelet/pods/abb24fe7-ab6b-4c77-b68a-cb471638f7a0/volumes"
Sep 06 19:22:47.185059 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:47.185025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:47.286077 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:47.286039    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:49.702555 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:49.702524    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:49.802605 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:49.802568    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:54.261466 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:54.261428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:54.361668 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:54.361640    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:54.461945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:54.461905    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:55.200207 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:55.200176    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-knvzq"
Sep 06 19:22:58.348860 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:58.348824    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:22:58.453349 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:22:58.453283    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:01.280272 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:01.280239    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:01.383355 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:01.383320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:06.089937 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:06.089854    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:06.185777 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:06.185741    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:08.775202 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:08.775169    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:08.877650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:08.877611    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:12.295926 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.295892    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"]
Sep 06 19:23:12.300116 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.300096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.302010 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.301987    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-sbffg\""
Sep 06 19:23:12.310443 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.310412    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"]
Sep 06 19:23:12.314622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.314573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:12.339993 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.339955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6327769-d8ea-4264-bc35-64351b5f1eb4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.340222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.340023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.340222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.340088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.340222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.340117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.340222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.340146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.340222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.340166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.340415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.340246    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmlcm\" (UniqueName: \"kubernetes.io/projected/d6327769-d8ea-4264-bc35-64351b5f1eb4-kube-api-access-bmlcm\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.340415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.340287    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.340415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.340313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.440832 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.440791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlcm\" (UniqueName: \"kubernetes.io/projected/d6327769-d8ea-4264-bc35-64351b5f1eb4-kube-api-access-bmlcm\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441058 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.440858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441058 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.440903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441058 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.440934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6327769-d8ea-4264-bc35-64351b5f1eb4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441058 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.440990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441058 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.441050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.441116    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.441156    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.441183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.441398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441807 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.441566    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441807 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.441639    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.441807 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.441658    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.442162 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.442131    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6327769-d8ea-4264-bc35-64351b5f1eb4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.444514 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.444488    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.445305 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.445287    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.448399 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.448370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmlcm\" (UniqueName: \"kubernetes.io/projected/d6327769-d8ea-4264-bc35-64351b5f1eb4-kube-api-access-bmlcm\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.448558 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.448542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6327769-d8ea-4264-bc35-64351b5f1eb4-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jh9s2\" (UID: \"d6327769-d8ea-4264-bc35-64351b5f1eb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.470054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.469961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:12.611878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.611826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:12.889059 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.889029    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"]
Sep 06 19:23:12.891273 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:23:12.891242    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6327769_d8ea_4264_bc35_64351b5f1eb4.slice/crio-d7c9fe6d70364f2526cbbd68ef122165cffd838b5ee715e416337fc25320bee5 WatchSource:0}: Error finding container d7c9fe6d70364f2526cbbd68ef122165cffd838b5ee715e416337fc25320bee5: Status 404 returned error can't find the container with id d7c9fe6d70364f2526cbbd68ef122165cffd838b5ee715e416337fc25320bee5
Sep 06 19:23:12.893995 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.893945    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:23:12.894122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.894016    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:23:12.894122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:12.894046    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:23:13.335928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:13.335831    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" event={"ID":"d6327769-d8ea-4264-bc35-64351b5f1eb4","Type":"ContainerStarted","Data":"d7cc66e859762dc052bde68b84249c2f50ad37ba6c638bb232a5a22a2ae7165e"}
Sep 06 19:23:13.335928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:13.335896    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" event={"ID":"d6327769-d8ea-4264-bc35-64351b5f1eb4","Type":"ContainerStarted","Data":"d7c9fe6d70364f2526cbbd68ef122165cffd838b5ee715e416337fc25320bee5"}
Sep 06 19:23:13.353676 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:13.353615    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" podStartSLOduration=1.353596267 podStartE2EDuration="1.353596267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:23:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:23:13.351822338 +0000 UTC m=+721.632275135" watchObservedRunningTime="2026-09-06 19:23:13.353596267 +0000 UTC m=+721.634049061"
Sep 06 19:23:13.612549 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:13.612446    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:13.617616 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:13.617585    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:14.093771 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:14.093738    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:14.194366 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:14.194333    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:14.340462 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:14.340432    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:14.341540 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:14.341522    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:23:14.493493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:14.493458    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:14.595207 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:14.595173    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:14.892722 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:14.892694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:14.949246 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:14.949217    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:21.105358 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:21.105311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:21.201855 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:21.201816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:25.144902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:25.144870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:25.251972 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:25.251918    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:25.307175 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:25.307129    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:25.916676 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:25.916635    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:26.025770 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.025735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:26.147330 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.147293    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bq24b"]
Sep 06 19:23:26.152209 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.152176    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-bq24b"
Sep 06 19:23:26.154166 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.154142    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8fwdc\""
Sep 06 19:23:26.156615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.156576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:26.157564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.157539    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bq24b"]
Sep 06 19:23:26.221911 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.221807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:26.277271 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.277232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtvkt\" (UniqueName: \"kubernetes.io/projected/7487bb59-d343-4bfb-9bb0-c775aaba07fc-kube-api-access-xtvkt\") pod \"authorino-c548b5d7d-bq24b\" (UID: \"7487bb59-d343-4bfb-9bb0-c775aaba07fc\") " pod="kuadrant-system/authorino-c548b5d7d-bq24b"
Sep 06 19:23:26.378034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.377984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtvkt\" (UniqueName: \"kubernetes.io/projected/7487bb59-d343-4bfb-9bb0-c775aaba07fc-kube-api-access-xtvkt\") pod \"authorino-c548b5d7d-bq24b\" (UID: \"7487bb59-d343-4bfb-9bb0-c775aaba07fc\") " pod="kuadrant-system/authorino-c548b5d7d-bq24b"
Sep 06 19:23:26.386240 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.386212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtvkt\" (UniqueName: \"kubernetes.io/projected/7487bb59-d343-4bfb-9bb0-c775aaba07fc-kube-api-access-xtvkt\") pod \"authorino-c548b5d7d-bq24b\" (UID: \"7487bb59-d343-4bfb-9bb0-c775aaba07fc\") " pod="kuadrant-system/authorino-c548b5d7d-bq24b"
Sep 06 19:23:26.478749 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.478656    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-bq24b"
Sep 06 19:23:26.748555 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:26.748423    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bq24b"]
Sep 06 19:23:26.751181 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:23:26.751147    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7487bb59_d343_4bfb_9bb0_c775aaba07fc.slice/crio-162496346b9fe32b4b34e2ee0c75c61728cd5e2faf2bcb0d98810e2ba289e949 WatchSource:0}: Error finding container 162496346b9fe32b4b34e2ee0c75c61728cd5e2faf2bcb0d98810e2ba289e949: Status 404 returned error can't find the container with id 162496346b9fe32b4b34e2ee0c75c61728cd5e2faf2bcb0d98810e2ba289e949
Sep 06 19:23:27.409338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:27.409304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-bq24b" event={"ID":"7487bb59-d343-4bfb-9bb0-c775aaba07fc","Type":"ContainerStarted","Data":"162496346b9fe32b4b34e2ee0c75c61728cd5e2faf2bcb0d98810e2ba289e949"}
Sep 06 19:23:29.421744 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:29.421698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-bq24b" event={"ID":"7487bb59-d343-4bfb-9bb0-c775aaba07fc","Type":"ContainerStarted","Data":"82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1"}
Sep 06 19:23:29.435690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:29.435626    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-bq24b" podStartSLOduration=1.493753316 podStartE2EDuration="3.435608755s" totalImagesPullingTime="1.941855439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:23:29.434287303 +0000 UTC m=+737.714740098" watchObservedRunningTime="2026-09-06 19:23:29.435608755 +0000 UTC m=+737.716061553"
Sep 06 19:23:45.736326 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:45.736290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:45.836537 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:45.836497    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:46.201560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:46.201528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:58.586945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:58.586904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:58.815200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:58.815167    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-ktgrx"]
Sep 06 19:23:58.819942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:58.819919    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx"
Sep 06 19:23:58.825391 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:58.825362    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-ktgrx"]
Sep 06 19:23:58.876737 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:58.876643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8klc9\" (UniqueName: \"kubernetes.io/projected/2d8ea2aa-6667-4abe-9333-3174d27c863d-kube-api-access-8klc9\") pod \"authorino-5dc7ccb5f-ktgrx\" (UID: \"2d8ea2aa-6667-4abe-9333-3174d27c863d\") " pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx"
Sep 06 19:23:58.904630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:58.904589    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:23:58.977868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:58.977822    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8klc9\" (UniqueName: \"kubernetes.io/projected/2d8ea2aa-6667-4abe-9333-3174d27c863d-kube-api-access-8klc9\") pod \"authorino-5dc7ccb5f-ktgrx\" (UID: \"2d8ea2aa-6667-4abe-9333-3174d27c863d\") " pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx"
Sep 06 19:23:58.986296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:58.986266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8klc9\" (UniqueName: \"kubernetes.io/projected/2d8ea2aa-6667-4abe-9333-3174d27c863d-kube-api-access-8klc9\") pod \"authorino-5dc7ccb5f-ktgrx\" (UID: \"2d8ea2aa-6667-4abe-9333-3174d27c863d\") " pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx"
Sep 06 19:23:59.033599 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.033560    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-ktgrx"]
Sep 06 19:23:59.033892 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.033877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx"
Sep 06 19:23:59.059407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.059355    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76cb8d85d7-zzs8l"]
Sep 06 19:23:59.071621 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.071325    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76cb8d85d7-zzs8l"
Sep 06 19:23:59.072342 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.072314    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76cb8d85d7-zzs8l"]
Sep 06 19:23:59.101490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.101451    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76cb8d85d7-zzs8l"]
Sep 06 19:23:59.101809 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:23:59.101782    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2vxqc], unattached volumes=[], failed to process volumes=[kube-api-access-2vxqc]: context canceled" pod="kuadrant-system/authorino-76cb8d85d7-zzs8l" podUID="727fabe6-0c3d-4da3-b7c0-be6fa8b88b50"
Sep 06 19:23:59.126042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.125969    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7f9c688c49-9rgqd"]
Sep 06 19:23:59.129882 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.129823    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:23:59.131988 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.131965    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 06 19:23:59.135995 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.135937    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f9c688c49-9rgqd"]
Sep 06 19:23:59.181625 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.181592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqkgh\" (UniqueName: \"kubernetes.io/projected/2992fde4-7ebf-491d-ac50-f7e397edc3cc-kube-api-access-qqkgh\") pod \"authorino-7f9c688c49-9rgqd\" (UID: \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\") " pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:23:59.181803 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.181650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vxqc\" (UniqueName: \"kubernetes.io/projected/727fabe6-0c3d-4da3-b7c0-be6fa8b88b50-kube-api-access-2vxqc\") pod \"authorino-76cb8d85d7-zzs8l\" (UID: \"727fabe6-0c3d-4da3-b7c0-be6fa8b88b50\") " pod="kuadrant-system/authorino-76cb8d85d7-zzs8l"
Sep 06 19:23:59.181803 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.181686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2992fde4-7ebf-491d-ac50-f7e397edc3cc-tls-cert\") pod \"authorino-7f9c688c49-9rgqd\" (UID: \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\") " pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:23:59.283194 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.283149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqkgh\" (UniqueName: \"kubernetes.io/projected/2992fde4-7ebf-491d-ac50-f7e397edc3cc-kube-api-access-qqkgh\") pod \"authorino-7f9c688c49-9rgqd\" (UID: \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\") " pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:23:59.283403 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.283212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vxqc\" (UniqueName: \"kubernetes.io/projected/727fabe6-0c3d-4da3-b7c0-be6fa8b88b50-kube-api-access-2vxqc\") pod \"authorino-76cb8d85d7-zzs8l\" (UID: \"727fabe6-0c3d-4da3-b7c0-be6fa8b88b50\") " pod="kuadrant-system/authorino-76cb8d85d7-zzs8l"
Sep 06 19:23:59.283403 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.283240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2992fde4-7ebf-491d-ac50-f7e397edc3cc-tls-cert\") pod \"authorino-7f9c688c49-9rgqd\" (UID: \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\") " pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:23:59.285843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.285820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2992fde4-7ebf-491d-ac50-f7e397edc3cc-tls-cert\") pod \"authorino-7f9c688c49-9rgqd\" (UID: \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\") " pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:23:59.290297 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.290267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vxqc\" (UniqueName: \"kubernetes.io/projected/727fabe6-0c3d-4da3-b7c0-be6fa8b88b50-kube-api-access-2vxqc\") pod \"authorino-76cb8d85d7-zzs8l\" (UID: \"727fabe6-0c3d-4da3-b7c0-be6fa8b88b50\") " pod="kuadrant-system/authorino-76cb8d85d7-zzs8l"
Sep 06 19:23:59.290458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.290436    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqkgh\" (UniqueName: \"kubernetes.io/projected/2992fde4-7ebf-491d-ac50-f7e397edc3cc-kube-api-access-qqkgh\") pod \"authorino-7f9c688c49-9rgqd\" (UID: \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\") " pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:23:59.313686 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.313653    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-ktgrx"]
Sep 06 19:23:59.315762 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:23:59.315728    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d8ea2aa_6667_4abe_9333_3174d27c863d.slice/crio-cf2037bdbafbc02de965f16a1e656a0ba1913300c8340b7ce4bfeafbf76a23c2 WatchSource:0}: Error finding container cf2037bdbafbc02de965f16a1e656a0ba1913300c8340b7ce4bfeafbf76a23c2: Status 404 returned error can't find the container with id cf2037bdbafbc02de965f16a1e656a0ba1913300c8340b7ce4bfeafbf76a23c2
Sep 06 19:23:59.441372 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.441335    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:23:59.576144 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.576111    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx" event={"ID":"2d8ea2aa-6667-4abe-9333-3174d27c863d","Type":"ContainerStarted","Data":"cf2037bdbafbc02de965f16a1e656a0ba1913300c8340b7ce4bfeafbf76a23c2"}
Sep 06 19:23:59.576144 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.576127    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76cb8d85d7-zzs8l"
Sep 06 19:23:59.584354 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.584256    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76cb8d85d7-zzs8l"
Sep 06 19:23:59.687418 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.687380    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vxqc\" (UniqueName: \"kubernetes.io/projected/727fabe6-0c3d-4da3-b7c0-be6fa8b88b50-kube-api-access-2vxqc\") pod \"727fabe6-0c3d-4da3-b7c0-be6fa8b88b50\" (UID: \"727fabe6-0c3d-4da3-b7c0-be6fa8b88b50\") "
Sep 06 19:23:59.689573 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.689548    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/727fabe6-0c3d-4da3-b7c0-be6fa8b88b50-kube-api-access-2vxqc" (OuterVolumeSpecName: "kube-api-access-2vxqc") pod "727fabe6-0c3d-4da3-b7c0-be6fa8b88b50" (UID: "727fabe6-0c3d-4da3-b7c0-be6fa8b88b50"). InnerVolumeSpecName "kube-api-access-2vxqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:23:59.713709 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.713680    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f9c688c49-9rgqd"]
Sep 06 19:23:59.715737 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:23:59.715712    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2992fde4_7ebf_491d_ac50_f7e397edc3cc.slice/crio-fcd88db8be2507797642e5c14a280aaf9e4e9dc4c0b45e26266a87d945e34eab WatchSource:0}: Error finding container fcd88db8be2507797642e5c14a280aaf9e4e9dc4c0b45e26266a87d945e34eab: Status 404 returned error can't find the container with id fcd88db8be2507797642e5c14a280aaf9e4e9dc4c0b45e26266a87d945e34eab
Sep 06 19:23:59.788514 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:23:59.788476    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vxqc\" (UniqueName: \"kubernetes.io/projected/727fabe6-0c3d-4da3-b7c0-be6fa8b88b50-kube-api-access-2vxqc\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:24:00.583316 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.583212    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f9c688c49-9rgqd" event={"ID":"2992fde4-7ebf-491d-ac50-f7e397edc3cc","Type":"ContainerStarted","Data":"d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248"}
Sep 06 19:24:00.583316 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.583264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f9c688c49-9rgqd" event={"ID":"2992fde4-7ebf-491d-ac50-f7e397edc3cc","Type":"ContainerStarted","Data":"fcd88db8be2507797642e5c14a280aaf9e4e9dc4c0b45e26266a87d945e34eab"}
Sep 06 19:24:00.584832 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.584805    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx" event={"ID":"2d8ea2aa-6667-4abe-9333-3174d27c863d","Type":"ContainerStarted","Data":"4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947"}
Sep 06 19:24:00.584979 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.584835    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76cb8d85d7-zzs8l"
Sep 06 19:24:00.584979 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.584845    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx" podUID="2d8ea2aa-6667-4abe-9333-3174d27c863d" containerName="authorino" containerID="cri-o://4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947" gracePeriod=30
Sep 06 19:24:00.598040 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.597992    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7f9c688c49-9rgqd" podStartSLOduration=1.222778543 podStartE2EDuration="1.597976948s" totalImagesPullingTime="375.198405ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:23:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:24:00.596148468 +0000 UTC m=+768.876601262" watchObservedRunningTime="2026-09-06 19:24:00.597976948 +0000 UTC m=+768.878429773"
Sep 06 19:24:00.617634 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.617605    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76cb8d85d7-zzs8l"]
Sep 06 19:24:00.624495 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.624468    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76cb8d85d7-zzs8l"]
Sep 06 19:24:00.627332 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.627304    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bq24b"]
Sep 06 19:24:00.627538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.627513    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-bq24b" podUID="7487bb59-d343-4bfb-9bb0-c775aaba07fc" containerName="authorino" containerID="cri-o://82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1" gracePeriod=30
Sep 06 19:24:00.635318 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.635080    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx" podStartSLOduration=1.989175178 podStartE2EDuration="2.635041584s" totalImagesPullingTime="645.866406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:24:00.634487375 +0000 UTC m=+768.914940170" watchObservedRunningTime="2026-09-06 19:24:00.635041584 +0000 UTC m=+768.915494378"
Sep 06 19:24:00.888219 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.888192    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx"
Sep 06 19:24:00.898811 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.898784    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-bq24b"
Sep 06 19:24:00.999238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.999199    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8klc9\" (UniqueName: \"kubernetes.io/projected/2d8ea2aa-6667-4abe-9333-3174d27c863d-kube-api-access-8klc9\") pod \"2d8ea2aa-6667-4abe-9333-3174d27c863d\" (UID: \"2d8ea2aa-6667-4abe-9333-3174d27c863d\") "
Sep 06 19:24:00.999407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:00.999258    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtvkt\" (UniqueName: \"kubernetes.io/projected/7487bb59-d343-4bfb-9bb0-c775aaba07fc-kube-api-access-xtvkt\") pod \"7487bb59-d343-4bfb-9bb0-c775aaba07fc\" (UID: \"7487bb59-d343-4bfb-9bb0-c775aaba07fc\") "
Sep 06 19:24:01.001607 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.001573    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7487bb59-d343-4bfb-9bb0-c775aaba07fc-kube-api-access-xtvkt" (OuterVolumeSpecName: "kube-api-access-xtvkt") pod "7487bb59-d343-4bfb-9bb0-c775aaba07fc" (UID: "7487bb59-d343-4bfb-9bb0-c775aaba07fc"). InnerVolumeSpecName "kube-api-access-xtvkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:24:01.001607 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.001571    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8ea2aa-6667-4abe-9333-3174d27c863d-kube-api-access-8klc9" (OuterVolumeSpecName: "kube-api-access-8klc9") pod "2d8ea2aa-6667-4abe-9333-3174d27c863d" (UID: "2d8ea2aa-6667-4abe-9333-3174d27c863d"). InnerVolumeSpecName "kube-api-access-8klc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:24:01.100462 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.100379    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8klc9\" (UniqueName: \"kubernetes.io/projected/2d8ea2aa-6667-4abe-9333-3174d27c863d-kube-api-access-8klc9\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:24:01.100462 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.100408    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtvkt\" (UniqueName: \"kubernetes.io/projected/7487bb59-d343-4bfb-9bb0-c775aaba07fc-kube-api-access-xtvkt\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:24:01.591381 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.591346    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d8ea2aa-6667-4abe-9333-3174d27c863d" containerID="4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947" exitCode=0
Sep 06 19:24:01.591570 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.591388    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx"
Sep 06 19:24:01.591570 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.591444    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx" event={"ID":"2d8ea2aa-6667-4abe-9333-3174d27c863d","Type":"ContainerDied","Data":"4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947"}
Sep 06 19:24:01.591570 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.591488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-ktgrx" event={"ID":"2d8ea2aa-6667-4abe-9333-3174d27c863d","Type":"ContainerDied","Data":"cf2037bdbafbc02de965f16a1e656a0ba1913300c8340b7ce4bfeafbf76a23c2"}
Sep 06 19:24:01.591570 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.591513    2581 scope.go:117] "RemoveContainer" containerID="4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947"
Sep 06 19:24:01.592799 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.592764    2581 generic.go:356] "Generic (PLEG): container finished" podID="7487bb59-d343-4bfb-9bb0-c775aaba07fc" containerID="82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1" exitCode=0
Sep 06 19:24:01.592876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.592797    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-bq24b" event={"ID":"7487bb59-d343-4bfb-9bb0-c775aaba07fc","Type":"ContainerDied","Data":"82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1"}
Sep 06 19:24:01.592876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.592825    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-bq24b"
Sep 06 19:24:01.592876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.592834    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-bq24b" event={"ID":"7487bb59-d343-4bfb-9bb0-c775aaba07fc","Type":"ContainerDied","Data":"162496346b9fe32b4b34e2ee0c75c61728cd5e2faf2bcb0d98810e2ba289e949"}
Sep 06 19:24:01.601484 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.601463    2581 scope.go:117] "RemoveContainer" containerID="4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947"
Sep 06 19:24:01.601867 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:24:01.601845    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947\": container with ID starting with 4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947 not found: ID does not exist" containerID="4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947"
Sep 06 19:24:01.601970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.601876    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947"} err="failed to get container status \"4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947\": rpc error: code = NotFound desc = could not find container \"4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947\": container with ID starting with 4aa52d707a2cca3bce3d2ce5d0d54cd27c8ece9528b58ecdace477550d433947 not found: ID does not exist"
Sep 06 19:24:01.601970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.601900    2581 scope.go:117] "RemoveContainer" containerID="82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1"
Sep 06 19:24:01.610864 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.610843    2581 scope.go:117] "RemoveContainer" containerID="82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1"
Sep 06 19:24:01.611198 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:24:01.611178    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1\": container with ID starting with 82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1 not found: ID does not exist" containerID="82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1"
Sep 06 19:24:01.611244 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.611207    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1"} err="failed to get container status \"82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1\": rpc error: code = NotFound desc = could not find container \"82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1\": container with ID starting with 82856227fb00704abe3ebf3d26d2aa5f9e1d2b43df174cfbd93111bf52bdeeb1 not found: ID does not exist"
Sep 06 19:24:01.614127 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.614058    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bq24b"]
Sep 06 19:24:01.615868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.615845    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bq24b"]
Sep 06 19:24:01.624158 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.624130    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-ktgrx"]
Sep 06 19:24:01.629711 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:01.629689    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-ktgrx"]
Sep 06 19:24:02.273279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:02.273234    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d8ea2aa-6667-4abe-9333-3174d27c863d" path="/var/lib/kubelet/pods/2d8ea2aa-6667-4abe-9333-3174d27c863d/volumes"
Sep 06 19:24:02.273773 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:02.273717    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="727fabe6-0c3d-4da3-b7c0-be6fa8b88b50" path="/var/lib/kubelet/pods/727fabe6-0c3d-4da3-b7c0-be6fa8b88b50/volumes"
Sep 06 19:24:02.273977 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:02.273958    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7487bb59-d343-4bfb-9bb0-c775aaba07fc" path="/var/lib/kubelet/pods/7487bb59-d343-4bfb-9bb0-c775aaba07fc/volumes"
Sep 06 19:24:05.150084 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:05.150032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:05.236405 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:05.236372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:05.289486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:05.289453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:05.414259 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:05.414174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:05.514543 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:05.514509    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:07.017455 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:07.017421    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:07.067754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:07.067721    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:08.520389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:08.520342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:08.614218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:08.614178    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:38.569924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:38.569878    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:38.668935 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:38.668896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:40.057579 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:40.057547    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:40.121717 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:40.121685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:40.290625 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:40.290597    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:40.391037 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:40.390985    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:40.625455 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:40.625408    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:40.730008 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:40.729918    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:44.588178 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:44.588140    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:44.640975 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:44.640943    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:50.858827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:50.858794    2581 generic.go:356] "Generic (PLEG): container finished" podID="d6327769-d8ea-4264-bc35-64351b5f1eb4" containerID="d7cc66e859762dc052bde68b84249c2f50ad37ba6c638bb232a5a22a2ae7165e" exitCode=0
Sep 06 19:24:50.859279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:50.858867    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" event={"ID":"d6327769-d8ea-4264-bc35-64351b5f1eb4","Type":"ContainerDied","Data":"d7cc66e859762dc052bde68b84249c2f50ad37ba6c638bb232a5a22a2ae7165e"}
Sep 06 19:24:50.859335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:50.859325    2581 scope.go:117] "RemoveContainer" containerID="d7cc66e859762dc052bde68b84249c2f50ad37ba6c638bb232a5a22a2ae7165e"
Sep 06 19:24:50.860312 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:50.860287    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:24:50.860372 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:50.860352    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:24:50.860407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:50.860388    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:24:51.466635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:51.466597    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:51.566955 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:51.566920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:24:51.866464 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:51.866380    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" event={"ID":"d6327769-d8ea-4264-bc35-64351b5f1eb4","Type":"ContainerStarted","Data":"e6ddcf7d192fdef002d3b97f60491bf74ad9e2322730434bc7fa422aed65fa29"}
Sep 06 19:24:52.612377 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:52.612327    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:24:52.612551 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:52.612467    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jh9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 06 19:24:52.612551 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:52.612513    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" podUID="d6327769-d8ea-4264-bc35-64351b5f1eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 06 19:24:53.612528 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:53.612488    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jh9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 06 19:24:53.612911 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:53.612562    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" podUID="d6327769-d8ea-4264-bc35-64351b5f1eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 06 19:24:54.612651 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:54.612597    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jh9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 06 19:24:54.613173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:54.612708    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" podUID="d6327769-d8ea-4264-bc35-64351b5f1eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 06 19:24:55.612151 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:55.612113    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jh9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 06 19:24:55.612358 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:55.612181    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" podUID="d6327769-d8ea-4264-bc35-64351b5f1eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 06 19:24:56.613734 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:56.613341    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jh9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 06 19:24:56.614240 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:56.613765    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2" podUID="d6327769-d8ea-4264-bc35-64351b5f1eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 06 19:24:57.616512 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:57.616476    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:24:57.616869 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:57.616785    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:24:57.617626 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:24:57.617606    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jh9s2"
Sep 06 19:25:01.770839 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:01.770804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:25:11.264627 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:11.264598    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:25:14.990415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:14.990382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:25:15.041802 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:15.041766    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:25:16.535558 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.535518    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"]
Sep 06 19:25:16.536632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.535906    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7487bb59-d343-4bfb-9bb0-c775aaba07fc" containerName="authorino"
Sep 06 19:25:16.536632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.535917    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7487bb59-d343-4bfb-9bb0-c775aaba07fc" containerName="authorino"
Sep 06 19:25:16.536632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.535940    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8ea2aa-6667-4abe-9333-3174d27c863d" containerName="authorino"
Sep 06 19:25:16.536632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.535945    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8ea2aa-6667-4abe-9333-3174d27c863d" containerName="authorino"
Sep 06 19:25:16.536632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.536005    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7487bb59-d343-4bfb-9bb0-c775aaba07fc" containerName="authorino"
Sep 06 19:25:16.536632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.536018    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8ea2aa-6667-4abe-9333-3174d27c863d" containerName="authorino"
Sep 06 19:25:16.539891 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.539870    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.542051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.542027    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 06 19:25:16.542755 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.542727    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 06 19:25:16.542880 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.542752    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-84kjj\""
Sep 06 19:25:16.542880 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.542814    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 06 19:25:16.547903 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.547868    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"]
Sep 06 19:25:16.572102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.572040    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:25:16.610328 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.610298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.610328 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.610332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b96d2c19-5f2a-499a-ba00-2b1018db2419-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.610548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.610423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.610548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.610454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.610548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.610473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.610644 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.610560    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d4vr\" (UniqueName: \"kubernetes.io/projected/b96d2c19-5f2a-499a-ba00-2b1018db2419-kube-api-access-8d4vr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.711253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.711207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.711253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.711250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b96d2c19-5f2a-499a-ba00-2b1018db2419-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.711543 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.711409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.711543 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.711445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.711543 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.711465    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.711733 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.711706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.711934 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.711911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d4vr\" (UniqueName: \"kubernetes.io/projected/b96d2c19-5f2a-499a-ba00-2b1018db2419-kube-api-access-8d4vr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.712026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.711992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.712256 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.712236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.714237 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.714207    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b96d2c19-5f2a-499a-ba00-2b1018db2419-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.714710 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.714688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b96d2c19-5f2a-499a-ba00-2b1018db2419-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.719475 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.719450    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d4vr\" (UniqueName: \"kubernetes.io/projected/b96d2c19-5f2a-499a-ba00-2b1018db2419-kube-api-access-8d4vr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv\" (UID: \"b96d2c19-5f2a-499a-ba00-2b1018db2419\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:16.854219 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:16.854125    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:17.130189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:17.129973    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"]
Sep 06 19:25:17.132437 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:25:17.132404    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96d2c19_5f2a_499a_ba00_2b1018db2419.slice/crio-c4481160b12c3c41c2ff896c796a4e3e8777af15a9633773f3f34ec22079b85b WatchSource:0}: Error finding container c4481160b12c3c41c2ff896c796a4e3e8777af15a9633773f3f34ec22079b85b: Status 404 returned error can't find the container with id c4481160b12c3c41c2ff896c796a4e3e8777af15a9633773f3f34ec22079b85b
Sep 06 19:25:18.004302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:18.004264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv" event={"ID":"b96d2c19-5f2a-499a-ba00-2b1018db2419","Type":"ContainerStarted","Data":"c4481160b12c3c41c2ff896c796a4e3e8777af15a9633773f3f34ec22079b85b"}
Sep 06 19:25:19.011564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:19.011524    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv" event={"ID":"b96d2c19-5f2a-499a-ba00-2b1018db2419","Type":"ContainerStarted","Data":"7348c951445774f319e0f7c4ad0f045479fc8463c98272c9a702fc58785645f3"}
Sep 06 19:25:19.027811 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:19.027753    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv" podStartSLOduration=1.944575028 podStartE2EDuration="3.027735309s" totalImagesPullingTime="1.083160281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:25:19.025932121 +0000 UTC m=+847.306384916" watchObservedRunningTime="2026-09-06 19:25:19.027735309 +0000 UTC m=+847.308188102"
Sep 06 19:25:26.854771 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:26.854723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:26.855195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:26.854813    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:26.867503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:26.867476    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:27.062614 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:27.062585    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv"
Sep 06 19:25:27.573461 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:27.573426    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:25:36.140641 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.140600    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"]
Sep 06 19:25:36.144474 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.144448    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.149983 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.149958    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 06 19:25:36.158286 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.158256    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"]
Sep 06 19:25:36.196840 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.196804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:25:36.312883 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.312831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.313177 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.312895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.313177 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.312926    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b875h\" (UniqueName: \"kubernetes.io/projected/021fe78f-d323-4e97-b01c-3ee387b330aa-kube-api-access-b875h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.313177 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.312956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021fe78f-d323-4e97-b01c-3ee387b330aa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.313177 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.313036    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.313177 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.313106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.414029 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.413928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.414029 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.413967    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b875h\" (UniqueName: \"kubernetes.io/projected/021fe78f-d323-4e97-b01c-3ee387b330aa-kube-api-access-b875h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.414029 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.413996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021fe78f-d323-4e97-b01c-3ee387b330aa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.414357 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.414098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.414357 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.414290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.414522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.414503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.414596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.414527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.414660 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.414618    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.414802 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.414784    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.416490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.416468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021fe78f-d323-4e97-b01c-3ee387b330aa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.416777 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.416759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021fe78f-d323-4e97-b01c-3ee387b330aa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.421612 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.421593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b875h\" (UniqueName: \"kubernetes.io/projected/021fe78f-d323-4e97-b01c-3ee387b330aa-kube-api-access-b875h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4\" (UID: \"021fe78f-d323-4e97-b01c-3ee387b330aa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.454676 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.454644    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:36.727703 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.727671    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"]
Sep 06 19:25:36.729506 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:25:36.729466    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod021fe78f_d323_4e97_b01c_3ee387b330aa.slice/crio-a4bb5ab5b7758a9a713c9f178c325cb94cf3dca24924c775eeebded915f5368c WatchSource:0}: Error finding container a4bb5ab5b7758a9a713c9f178c325cb94cf3dca24924c775eeebded915f5368c: Status 404 returned error can't find the container with id a4bb5ab5b7758a9a713c9f178c325cb94cf3dca24924c775eeebded915f5368c
Sep 06 19:25:36.731010 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:36.730985    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 19:25:37.104746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:37.104710    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4" event={"ID":"021fe78f-d323-4e97-b01c-3ee387b330aa","Type":"ContainerStarted","Data":"0761998bd6dcfbf4a1b992901532688fba6bda8cfaf799c227b53fd90005b137"}
Sep 06 19:25:37.104927 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:37.104759    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4" event={"ID":"021fe78f-d323-4e97-b01c-3ee387b330aa","Type":"ContainerStarted","Data":"a4bb5ab5b7758a9a713c9f178c325cb94cf3dca24924c775eeebded915f5368c"}
Sep 06 19:25:37.120830 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:37.120771    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4" podStartSLOduration=0.931643328 podStartE2EDuration="1.120750628s" totalImagesPullingTime="189.1073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:25:37.118671441 +0000 UTC m=+865.399124236" watchObservedRunningTime="2026-09-06 19:25:37.120750628 +0000 UTC m=+865.401203421"
Sep 06 19:25:46.455372 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:46.455333    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:46.455770 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:46.455397    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:46.468310 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:46.468280    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:47.168992 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:47.168954    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4"
Sep 06 19:25:47.780573 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:47.780541    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:25:47.827650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:47.827616    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:25:51.575884 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:25:51.575845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:26:04.834445 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.834351    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"]
Sep 06 19:26:04.838909 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.838885    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.841009 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.840983    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 06 19:26:04.848816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.848787    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"]
Sep 06 19:26:04.870425 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.870397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.870575 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.870434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56d030f-115b-4e32-a220-46de8655828e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.870575 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.870466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.870665 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.870576    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwxhb\" (UniqueName: \"kubernetes.io/projected/f56d030f-115b-4e32-a220-46de8655828e-kube-api-access-qwxhb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.870665 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.870600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.870665 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.870638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.873625 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.873595    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:26:04.971713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.971680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwxhb\" (UniqueName: \"kubernetes.io/projected/f56d030f-115b-4e32-a220-46de8655828e-kube-api-access-qwxhb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.971713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.971718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.971958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.971745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.971958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.971790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.971958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.971819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56d030f-115b-4e32-a220-46de8655828e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.971958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.971838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.972214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.972166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.972260 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.972223    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.972296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.972253    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.974308 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.974285    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56d030f-115b-4e32-a220-46de8655828e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.974480 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.974461    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56d030f-115b-4e32-a220-46de8655828e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:04.979792 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:04.979762    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwxhb\" (UniqueName: \"kubernetes.io/projected/f56d030f-115b-4e32-a220-46de8655828e-kube-api-access-qwxhb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9rldw\" (UID: \"f56d030f-115b-4e32-a220-46de8655828e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:05.169704 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:05.169666    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:05.443043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:05.443015    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"]
Sep 06 19:26:05.445341 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:26:05.445319    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf56d030f_115b_4e32_a220_46de8655828e.slice/crio-b6c0e2d999af30afc064767ada0a3c86d4ce700c7afb64380f57ecb668d13904 WatchSource:0}: Error finding container b6c0e2d999af30afc064767ada0a3c86d4ce700c7afb64380f57ecb668d13904: Status 404 returned error can't find the container with id b6c0e2d999af30afc064767ada0a3c86d4ce700c7afb64380f57ecb668d13904
Sep 06 19:26:06.256130 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:06.256095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw" event={"ID":"f56d030f-115b-4e32-a220-46de8655828e","Type":"ContainerStarted","Data":"0803641e8bea3c4e35769e3ac27cfc36153525c9970422ca97dee72eb460b16f"}
Sep 06 19:26:06.256130 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:06.256138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw" event={"ID":"f56d030f-115b-4e32-a220-46de8655828e","Type":"ContainerStarted","Data":"b6c0e2d999af30afc064767ada0a3c86d4ce700c7afb64380f57ecb668d13904"}
Sep 06 19:26:06.272947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:06.272890    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw" podStartSLOduration=2.106555069 podStartE2EDuration="2.272873633s" totalImagesPullingTime="166.318564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:26:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:26:06.271404949 +0000 UTC m=+894.551857760" watchObservedRunningTime="2026-09-06 19:26:06.272873633 +0000 UTC m=+894.553326427"
Sep 06 19:26:12.240472 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:12.240441    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:26:12.241008 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:12.240988    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:26:12.246594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:12.246568    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:26:12.246795 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:12.246778    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:26:15.170545 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:15.170498    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:15.170971 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:15.170801    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:15.183456 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:15.183428    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:26:15.320737 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:26:15.320708    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9rldw"
Sep 06 19:27:09.537846 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:09.537813    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f9c688c49-9rgqd"]
Sep 06 19:27:09.538493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:09.538012    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7f9c688c49-9rgqd" podUID="2992fde4-7ebf-491d-ac50-f7e397edc3cc" containerName="authorino" containerID="cri-o://d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248" gracePeriod=30
Sep 06 19:27:09.791829 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:09.791768    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:27:09.890485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:09.890449    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2992fde4-7ebf-491d-ac50-f7e397edc3cc-tls-cert\") pod \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\" (UID: \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\") "
Sep 06 19:27:09.890666 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:09.890509    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqkgh\" (UniqueName: \"kubernetes.io/projected/2992fde4-7ebf-491d-ac50-f7e397edc3cc-kube-api-access-qqkgh\") pod \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\" (UID: \"2992fde4-7ebf-491d-ac50-f7e397edc3cc\") "
Sep 06 19:27:09.892689 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:09.892658    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2992fde4-7ebf-491d-ac50-f7e397edc3cc-kube-api-access-qqkgh" (OuterVolumeSpecName: "kube-api-access-qqkgh") pod "2992fde4-7ebf-491d-ac50-f7e397edc3cc" (UID: "2992fde4-7ebf-491d-ac50-f7e397edc3cc"). InnerVolumeSpecName "kube-api-access-qqkgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:27:09.901499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:09.901472    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2992fde4-7ebf-491d-ac50-f7e397edc3cc-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2992fde4-7ebf-491d-ac50-f7e397edc3cc" (UID: "2992fde4-7ebf-491d-ac50-f7e397edc3cc"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:27:09.992172 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:09.992137    2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2992fde4-7ebf-491d-ac50-f7e397edc3cc-tls-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:27:09.992172 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:09.992167    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqkgh\" (UniqueName: \"kubernetes.io/projected/2992fde4-7ebf-491d-ac50-f7e397edc3cc-kube-api-access-qqkgh\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:27:10.016504 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.016465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:10.110076 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.109967    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:10.604206 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.604172    2581 generic.go:356] "Generic (PLEG): container finished" podID="2992fde4-7ebf-491d-ac50-f7e397edc3cc" containerID="d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248" exitCode=0
Sep 06 19:27:10.604646 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.604211    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f9c688c49-9rgqd"
Sep 06 19:27:10.604646 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.604211    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f9c688c49-9rgqd" event={"ID":"2992fde4-7ebf-491d-ac50-f7e397edc3cc","Type":"ContainerDied","Data":"d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248"}
Sep 06 19:27:10.604646 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.604335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f9c688c49-9rgqd" event={"ID":"2992fde4-7ebf-491d-ac50-f7e397edc3cc","Type":"ContainerDied","Data":"fcd88db8be2507797642e5c14a280aaf9e4e9dc4c0b45e26266a87d945e34eab"}
Sep 06 19:27:10.604646 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.604358    2581 scope.go:117] "RemoveContainer" containerID="d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248"
Sep 06 19:27:10.619348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.619281    2581 scope.go:117] "RemoveContainer" containerID="d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248"
Sep 06 19:27:10.619718 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:27:10.619686    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248\": container with ID starting with d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248 not found: ID does not exist" containerID="d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248"
Sep 06 19:27:10.619787 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.619730    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248"} err="failed to get container status \"d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248\": rpc error: code = NotFound desc = could not find container \"d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248\": container with ID starting with d3bba0c860f8f20450659381143571564bebcec39b47f2cbac04507186379248 not found: ID does not exist"
Sep 06 19:27:10.620635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.620615    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f9c688c49-9rgqd"]
Sep 06 19:27:10.623565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:10.623545    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7f9c688c49-9rgqd"]
Sep 06 19:27:11.110662 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:11.110620    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:11.260875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:11.260840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:12.265965 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:12.265931    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2992fde4-7ebf-491d-ac50-f7e397edc3cc" path="/var/lib/kubelet/pods/2992fde4-7ebf-491d-ac50-f7e397edc3cc/volumes"
Sep 06 19:27:26.141503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.141468    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:26.257527 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.257485    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:26.428466 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.428378    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"]
Sep 06 19:27:26.428782 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.428767    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2992fde4-7ebf-491d-ac50-f7e397edc3cc" containerName="authorino"
Sep 06 19:27:26.428782 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.428781    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2992fde4-7ebf-491d-ac50-f7e397edc3cc" containerName="authorino"
Sep 06 19:27:26.428942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.428858    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2992fde4-7ebf-491d-ac50-f7e397edc3cc" containerName="authorino"
Sep 06 19:27:26.433723 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.433697    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.435741 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.435722    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-5307a7-openshift-default-dockercfg-rwzs9\""
Sep 06 19:27:26.443496 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.443470    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"]
Sep 06 19:27:26.449988 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.449958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:26.541758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.541718    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e562473-4e05-4539-8d95-b4c0a7439b13-istiod-ca-cert\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.541928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.541774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-socket\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.541928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.541797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-podinfo\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.541928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.541830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-envoy\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.541928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.541913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-credential-socket\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.542101 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.542003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-token\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.542101 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.542022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-certs\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.542101 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.542041    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-data\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.542217 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.542153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f66pc\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-kube-api-access-f66pc\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-envoy\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643387    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-credential-socket\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-token\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-certs\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-data\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f66pc\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-kube-api-access-f66pc\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e562473-4e05-4539-8d95-b4c0a7439b13-istiod-ca-cert\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-socket\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-podinfo\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.643983 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.643967    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-certs\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.644127 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.644108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-credential-socket\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.644212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.644146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-socket\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.644212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.644187    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-data\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.644972 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.644940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e562473-4e05-4539-8d95-b4c0a7439b13-istiod-ca-cert\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.645813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.645793    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-envoy\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.646388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.646363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-podinfo\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.650861 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.650830    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f66pc\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-kube-api-access-f66pc\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.651163 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.651143    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-token\") pod \"e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") " pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.747131 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.747037    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:26.912620 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:26.912583    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:27.032451 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.032419    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"]
Sep 06 19:27:27.034664 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:27:27.034635    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e562473_4e05_4539_8d95_b4c0a7439b13.slice/crio-50ff8ed0daa9dede55e00109931e024dc17f227a4f9ea466df437d2ad1d1ce5a WatchSource:0}: Error finding container 50ff8ed0daa9dede55e00109931e024dc17f227a4f9ea466df437d2ad1d1ce5a: Status 404 returned error can't find the container with id 50ff8ed0daa9dede55e00109931e024dc17f227a4f9ea466df437d2ad1d1ce5a
Sep 06 19:27:27.036794 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.036764    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:27.036895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.036828    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:27.036895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.036857    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:27.458839 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.458807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:27.548483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.548436    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:27.696129 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.696096    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" event={"ID":"6e562473-4e05-4539-8d95-b4c0a7439b13","Type":"ContainerStarted","Data":"53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d"}
Sep 06 19:27:27.696323 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.696146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" event={"ID":"6e562473-4e05-4539-8d95-b4c0a7439b13","Type":"ContainerStarted","Data":"50ff8ed0daa9dede55e00109931e024dc17f227a4f9ea466df437d2ad1d1ce5a"}
Sep 06 19:27:27.715540 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.715430    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" podStartSLOduration=1.715411968 podStartE2EDuration="1.715411968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:27.713443585 +0000 UTC m=+975.993896392" watchObservedRunningTime="2026-09-06 19:27:27.715411968 +0000 UTC m=+975.995864763"
Sep 06 19:27:27.747168 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.747138    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:27.916940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:27.916897    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:28.752784 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:28.752753    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:29.556317 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:29.556263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:29.660410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:29.660369    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:29.708875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:29.708846    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:29.709929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:29.709910    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:30.874695 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:30.874661    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:30.974921 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:30.974885    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:31.033311 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:31.033263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:31.330030 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:31.329991    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:31.406895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:31.406862    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:33.757874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:33.757766    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:33.858613 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:33.858576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:34.042782 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.042679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:34.244121 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.244048    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"]
Sep 06 19:27:34.248714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.248695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.251102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.251050    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0281bd63-openshift-default-dockercfg-dnqm4\""
Sep 06 19:27:34.256115 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.256082    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"]
Sep 06 19:27:34.276866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.276823    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:34.318179 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.318058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-istio-token\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.318179 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.318140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746e2bc6-528e-4d55-a445-c35121151ce5-istio-podinfo\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.318179 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.318167    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746e2bc6-528e-4d55-a445-c35121151ce5-istiod-ca-cert\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.318457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.318199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-credential-socket\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.318457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.318215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-data\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.318457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.318246    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2hfl\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-kube-api-access-s2hfl\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.318457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.318338    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-socket\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.318457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.318387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-certs\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.318688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.318475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-envoy\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.419594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.419552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-socket\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.419804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.419602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-certs\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.419804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.419622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-envoy\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.419804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.419666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-istio-token\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.419804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.419688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746e2bc6-528e-4d55-a445-c35121151ce5-istio-podinfo\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.419804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.419713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746e2bc6-528e-4d55-a445-c35121151ce5-istiod-ca-cert\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.419804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.419745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-credential-socket\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.419804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.419768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-data\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.419804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.419792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2hfl\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-kube-api-access-s2hfl\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.420279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.420038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-certs\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.420279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.420178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-socket\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.420389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.420304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-credential-socket\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.420389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.420372    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-data\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.420503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.420484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746e2bc6-528e-4d55-a445-c35121151ce5-istiod-ca-cert\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.422645 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.422622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-envoy\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.422915 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.422891    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746e2bc6-528e-4d55-a445-c35121151ce5-istio-podinfo\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.430227 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.430196    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-istio-token\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.430341 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.430262    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2hfl\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-kube-api-access-s2hfl\") pod \"e2e-ait-0281bd63-openshift-default-64f789b4-55l5z\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") " pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.441465 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.441437    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:34.562241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.562196    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:34.841947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.841916    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"]
Sep 06 19:27:34.844100 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:27:34.844050    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746e2bc6_528e_4d55_a445_c35121151ce5.slice/crio-daad61ed45bc5e87e7826d9cdef229a7f7e6afc89a97cf707dbc3684f4ce6df4 WatchSource:0}: Error finding container daad61ed45bc5e87e7826d9cdef229a7f7e6afc89a97cf707dbc3684f4ce6df4: Status 404 returned error can't find the container with id daad61ed45bc5e87e7826d9cdef229a7f7e6afc89a97cf707dbc3684f4ce6df4
Sep 06 19:27:34.849019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.846897    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:34.849019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.846987    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:34.849019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:34.847030    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:35.741045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:35.741007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" event={"ID":"746e2bc6-528e-4d55-a445-c35121151ce5","Type":"ContainerStarted","Data":"d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020"}
Sep 06 19:27:35.741247 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:35.741082    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" event={"ID":"746e2bc6-528e-4d55-a445-c35121151ce5","Type":"ContainerStarted","Data":"daad61ed45bc5e87e7826d9cdef229a7f7e6afc89a97cf707dbc3684f4ce6df4"}
Sep 06 19:27:35.763461 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:35.763402    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" podStartSLOduration=1.763380904 podStartE2EDuration="1.763380904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:35.761404533 +0000 UTC m=+984.041857352" watchObservedRunningTime="2026-09-06 19:27:35.763380904 +0000 UTC m=+984.043833700"
Sep 06 19:27:36.343000 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.342959    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:36.562975 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.562935    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:36.568304 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.568278    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:36.601302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.601226    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"]
Sep 06 19:27:36.604849 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.604828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.607029 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.607006    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-21b01e-openshift-default-dockercfg-6f44w\""
Sep 06 19:27:36.616731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.616703    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"]
Sep 06 19:27:36.631621 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.631594    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:36.641873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.641840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-credential-socket\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.642024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.641905    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-podinfo\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.642024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.641962    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkh2x\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-kube-api-access-xkh2x\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.642024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.642003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-envoy\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.642159 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.642047    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c070e3c-e087-4a77-88e5-db650bce4f57-istiod-ca-cert\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.642195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.642153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-data\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.642195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.642188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-certs\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.642259 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.642210    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-socket\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.642293 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.642256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-token\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.744415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.744376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-envoy\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.744712 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.744691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c070e3c-e087-4a77-88e5-db650bce4f57-istiod-ca-cert\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.744862 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.744848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-data\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.744963 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.744951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-certs\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.745089 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.745055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-socket\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.745230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.745217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-token\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.745356 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.745341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-credential-socket\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.745628 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.745610    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-podinfo\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.745816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.745792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkh2x\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-kube-api-access-xkh2x\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.747172 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.747112    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-socket\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.747727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.747707    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c070e3c-e087-4a77-88e5-db650bce4f57-istiod-ca-cert\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.747953 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.747935    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-data\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.748224 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.748202    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-certs\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.748652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.748633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-credential-socket\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.748961 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.748940    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:36.749047 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.749010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-envoy\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.750208 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.750189    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:36.752101 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.752078    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-podinfo\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.754097 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.754036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkh2x\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-kube-api-access-xkh2x\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.757684 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.757660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-token\") pod \"e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") " pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:36.917864 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.917823    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:36.918854 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:36.918831    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:37.133957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.133924    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:37.223783 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.223759    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"]
Sep 06 19:27:37.230019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.229984    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:37.230209 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.230094    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:37.230209 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.230140    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:37.247257 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.247225    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:37.755433 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.755400    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" event={"ID":"6c070e3c-e087-4a77-88e5-db650bce4f57","Type":"ContainerStarted","Data":"3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d"}
Sep 06 19:27:37.755874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.755460    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" event={"ID":"6c070e3c-e087-4a77-88e5-db650bce4f57","Type":"ContainerStarted","Data":"af58585856080bb117738679c935e9a0e5ea05f115740b55ec803417b2036ad4"}
Sep 06 19:27:37.776358 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.776296    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" podStartSLOduration=1.776276959 podStartE2EDuration="1.776276959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:27:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:37.774602275 +0000 UTC m=+986.055055098" watchObservedRunningTime="2026-09-06 19:27:37.776276959 +0000 UTC m=+986.056729752"
Sep 06 19:27:37.919604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:37.919558    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:38.279846 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:38.279815    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:38.383492 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:38.383451    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:38.481919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:38.481886    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:38.747299 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:38.747241    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:38.827301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:38.827268    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:38.924719 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:38.924688    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:39.637435 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:39.637399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:39.734037 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:39.734001    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:39.768004 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:39.767971    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:39.768986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:39.768970    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:27:40.772051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:40.771999    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:40.874844 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:40.874815    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:41.229691 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:41.229658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:41.331397 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:41.331363    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:44.572503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:44.572460    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:44.710846 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:44.710801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:44.805685 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:44.805649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:44.905098 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:44.905042    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:47.287437 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:47.287361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:47.387431 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:47.387388    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:47.558621 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:47.558531    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:47.632170 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:47.632124    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:49.045563 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:49.045532    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:51.676394 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:51.676362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:52.791651 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:52.791226    2581 patch_prober.go:28] interesting pod/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 19:27:52.791651 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:52.791301    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 19:27:53.143980 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.143946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:53.252294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.252249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:53.344798 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.344766    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:53.537955 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.537871    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:53.632249 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.632217    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"]
Sep 06 19:27:53.638638 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.635986    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.638638 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.638039    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-4067c966-kserve-self-signed-certs\""
Sep 06 19:27:53.645240 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.645216    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"]
Sep 06 19:27:53.736537 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.736497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-dshm\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.736737 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.736557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-model-cache\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.736737 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.736651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-home\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.736737 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.736690    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmh8f\" (UniqueName: \"kubernetes.io/projected/6ca801dd-7adb-4406-b0e8-2b1343365c4a-kube-api-access-zmh8f\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.736913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.736746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tmp-dir\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.736913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.736778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tls-certs\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.838873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.837619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-home\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.838873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.837678    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmh8f\" (UniqueName: \"kubernetes.io/projected/6ca801dd-7adb-4406-b0e8-2b1343365c4a-kube-api-access-zmh8f\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.838873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.837744    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tmp-dir\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.838873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.837775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tls-certs\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.838873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.837813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-dshm\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.838873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.837857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-model-cache\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.838873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.838341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-model-cache\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.838873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.838646    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tmp-dir\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.839749 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.838883    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-home\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.841075 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.841028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-dshm\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.841431 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.841407    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tls-certs\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.846972 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.846948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmh8f\" (UniqueName: \"kubernetes.io/projected/6ca801dd-7adb-4406-b0e8-2b1343365c4a-kube-api-access-zmh8f\") pod \"e2e-conflict-a-4067c966-kserve-574c788468-vv6k2\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") " pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:53.848348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.848323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:53.864713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.864675    2581 generic.go:356] "Generic (PLEG): container finished" podID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerID="53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d" exitCode=0
Sep 06 19:27:53.864896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.864733    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" event={"ID":"6e562473-4e05-4539-8d95-b4c0a7439b13","Type":"ContainerDied","Data":"53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d"}
Sep 06 19:27:53.865216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.865197    2581 scope.go:117] "RemoveContainer" containerID="53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d"
Sep 06 19:27:53.866467 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.866433    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:53.866603 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.866500    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:53.866603 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.866532    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:53.947279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:53.947247    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:27:54.269998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:54.269965    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"]
Sep 06 19:27:54.271174 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:27:54.271142    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ca801dd_7adb_4406_b0e8_2b1343365c4a.slice/crio-0d2664a4ee612edaa2cece3ec59309d281448b47b918a1b6b4729667610a59c6 WatchSource:0}: Error finding container 0d2664a4ee612edaa2cece3ec59309d281448b47b918a1b6b4729667610a59c6: Status 404 returned error can't find the container with id 0d2664a4ee612edaa2cece3ec59309d281448b47b918a1b6b4729667610a59c6
Sep 06 19:27:54.444389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:54.444321    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:54.740247 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:54.740208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:54.873994 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:54.873961    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" event={"ID":"6e562473-4e05-4539-8d95-b4c0a7439b13","Type":"ContainerStarted","Data":"8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6"}
Sep 06 19:27:54.875842 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:54.875819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2" event={"ID":"6ca801dd-7adb-4406-b0e8-2b1343365c4a","Type":"ContainerStarted","Data":"33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833"}
Sep 06 19:27:54.875950 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:54.875854    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2" event={"ID":"6ca801dd-7adb-4406-b0e8-2b1343365c4a","Type":"ContainerStarted","Data":"0d2664a4ee612edaa2cece3ec59309d281448b47b918a1b6b4729667610a59c6"}
Sep 06 19:27:54.911709 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:54.911630    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2" podStartSLOduration=1.911605987 podStartE2EDuration="1.911605987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:54.908747589 +0000 UTC m=+1003.189200385" watchObservedRunningTime="2026-09-06 19:27:54.911605987 +0000 UTC m=+1003.192058783"
Sep 06 19:27:55.023173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.020024    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"]
Sep 06 19:27:55.023173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.021661    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:55.049741 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.049681    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"]
Sep 06 19:27:55.049955 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.049821    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.055654 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.055620    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d73678-openshift-default-dockercfg-6vmvx\""
Sep 06 19:27:55.155764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.155623    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-data\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.155764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.155682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-podinfo\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.155764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.155717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-certs\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.156248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.155798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-socket\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.156248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.155901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-token\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.156248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.155940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-envoy\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.156248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.156038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw4cz\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-kube-api-access-pw4cz\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.156248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.156056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istiod-ca-cert\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.156248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.156133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-credential-socket\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.204891 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.204832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:55.256839 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.256809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-data\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.256870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-podinfo\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.256914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-certs\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.256960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-socket\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.257000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-token\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.257030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-envoy\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.257109    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw4cz\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-kube-api-access-pw4cz\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.257138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istiod-ca-cert\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.257187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-credential-socket\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.257274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-data\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.257604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.257505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-credential-socket\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.258302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.258026    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-socket\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.258499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.258453    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-certs\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.259110 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.259082    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istiod-ca-cert\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.259995 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.259972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-envoy\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.260279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.260259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-podinfo\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.266193 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.266117    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-token\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.266193 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.266156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw4cz\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-kube-api-access-pw4cz\") pod \"e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") " pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.365212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.365126    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:55.683600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.683575    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"]
Sep 06 19:27:55.694753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.693699    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:55.694753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.693794    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:55.694753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.693837    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:55.747715 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.747682    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:55.748799 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.748766    2581 patch_prober.go:28] interesting pod/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 19:27:55.748952 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.748933    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 19:27:55.885136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.885091    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" event={"ID":"2206bafb-7e52-48ec-9bb8-d661890a3a9f","Type":"ContainerStarted","Data":"4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36"}
Sep 06 19:27:55.885628 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.885166    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" event={"ID":"2206bafb-7e52-48ec-9bb8-d661890a3a9f","Type":"ContainerStarted","Data":"023d7648dec17a2b4aa743230e3bdda62a54d49665a82013d3dcb9fec87ed83b"}
Sep 06 19:27:55.890342 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.890300    2581 generic.go:356] "Generic (PLEG): container finished" podID="746e2bc6-528e-4d55-a445-c35121151ce5" containerID="d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020" exitCode=0
Sep 06 19:27:55.890757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.890496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" event={"ID":"746e2bc6-528e-4d55-a445-c35121151ce5","Type":"ContainerDied","Data":"d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020"}
Sep 06 19:27:55.891499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.891481    2581 scope.go:117] "RemoveContainer" containerID="d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020"
Sep 06 19:27:55.893005 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.892947    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:55.893143 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.893033    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:55.893143 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.893098    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:27:55.905847 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:55.905732    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" podStartSLOduration=1.905711291 podStartE2EDuration="1.905711291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:55.903151448 +0000 UTC m=+1004.183604270" watchObservedRunningTime="2026-09-06 19:27:55.905711291 +0000 UTC m=+1004.186164087"
Sep 06 19:27:56.367892 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:56.367800    2581 patch_prober.go:28] interesting pod/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 06 19:27:56.368184 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:56.368158    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 06 19:27:56.368321 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:56.367922    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:56.748356 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:56.748311    2581 patch_prober.go:28] interesting pod/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 19:27:56.748571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:56.748388    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 19:27:56.903527 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:56.903478    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" event={"ID":"746e2bc6-528e-4d55-a445-c35121151ce5","Type":"ContainerStarted","Data":"11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8"}
Sep 06 19:27:57.370152 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.370117    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:57.562940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.562903    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:27:57.563247 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.562959    2581 patch_prober.go:28] interesting pod/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 06 19:27:57.563350 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.563276    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 06 19:27:57.747934 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.747884    2581 patch_prober.go:28] interesting pod/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 19:27:57.748178 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.747962    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 19:27:57.761536 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.761491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:57.886830 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.886764    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:57.913531 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.913499    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:57.918320 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:57.918048    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:27:58.183373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:58.183321    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:58.563631 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:58.563524    2581 patch_prober.go:28] interesting pod/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 06 19:27:58.563631 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:58.563594    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 06 19:27:58.747748 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:58.747700    2581 patch_prober.go:28] interesting pod/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 19:27:58.747961 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:58.747776    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 19:27:58.915749 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:58.915711    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:59.011257 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:59.011161    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:59.190427 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:59.190275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:59.363402 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:59.363369    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:59.471313 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:59.471233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:27:59.563984 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:59.563454    2581 patch_prober.go:28] interesting pod/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 06 19:27:59.563984 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:59.563532    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 06 19:27:59.710179 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:59.709244    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:27:59.748349 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:59.748269    2581 patch_prober.go:28] interesting pod/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 19:27:59.748541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:27:59.748520    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 19:28:00.323385 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:00.323295    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:00.418848 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:00.418803    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:00.562771 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:00.562739    2581 patch_prober.go:28] interesting pod/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 06 19:28:00.562987 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:00.562791    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 06 19:28:00.753633 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:00.753597    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:28:00.754895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:00.754868    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:28:01.174231 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:01.174185    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:01.563238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:01.563148    2581 patch_prober.go:28] interesting pod/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 06 19:28:01.563238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:01.563204    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 06 19:28:01.793934 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:01.793872    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:02.391229 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.391191    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:02.535172 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.535126    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"]
Sep 06 19:28:02.540319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.540288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"
Sep 06 19:28:02.544723 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.543449    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:28:02.544723 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.543762    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:28:02.545767 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.545730    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"]
Sep 06 19:28:02.564758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.564717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179bd0b3-5af5-4c5b-9fe9-dff976512c7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0281bd63-c2928\" (UID: \"179bd0b3-5af5-4c5b-9fe9-dff976512c7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"
Sep 06 19:28:02.666887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.666797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179bd0b3-5af5-4c5b-9fe9-dff976512c7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0281bd63-c2928\" (UID: \"179bd0b3-5af5-4c5b-9fe9-dff976512c7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"
Sep 06 19:28:02.667778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.667738    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179bd0b3-5af5-4c5b-9fe9-dff976512c7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0281bd63-c2928\" (UID: \"179bd0b3-5af5-4c5b-9fe9-dff976512c7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"
Sep 06 19:28:02.806658 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.806606    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:28:02.807050 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.807027    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:28:02.808177 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.808145    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:28:02.865877 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:02.865838    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"
Sep 06 19:28:03.073045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.073009    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:03.168915 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.168885    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"]
Sep 06 19:28:03.171340 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:28:03.171308    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179bd0b3_5af5_4c5b_9fe9_dff976512c7f.slice/crio-addca0cc49346579f8c9f7c0a2c2396b3620b33b049d81bee43ebe4e0f2c6024 WatchSource:0}: Error finding container addca0cc49346579f8c9f7c0a2c2396b3620b33b049d81bee43ebe4e0f2c6024: Status 404 returned error can't find the container with id addca0cc49346579f8c9f7c0a2c2396b3620b33b049d81bee43ebe4e0f2c6024
Sep 06 19:28:03.726107 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.726040    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:03.948418 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.948382    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:28:03.948617 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.948442    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:28:03.961144 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.961117    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:28:03.981960 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.981870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928" event={"ID":"179bd0b3-5af5-4c5b-9fe9-dff976512c7f","Type":"ContainerStarted","Data":"addca0cc49346579f8c9f7c0a2c2396b3620b33b049d81bee43ebe4e0f2c6024"}
Sep 06 19:28:03.983509 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.983486    2581 generic.go:356] "Generic (PLEG): container finished" podID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerID="3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d" exitCode=0
Sep 06 19:28:03.983638 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.983564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" event={"ID":"6c070e3c-e087-4a77-88e5-db650bce4f57","Type":"ContainerDied","Data":"3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d"}
Sep 06 19:28:03.984092 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.984037    2581 scope.go:117] "RemoveContainer" containerID="3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d"
Sep 06 19:28:03.985191 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.985163    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:03.985296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.985247    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:03.985360 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.985293    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:03.995959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:03.995935    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:28:04.550318 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:04.550216    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:05.016639 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:05.016602    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" event={"ID":"6c070e3c-e087-4a77-88e5-db650bce4f57","Type":"ContainerStarted","Data":"1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b"}
Sep 06 19:28:05.919248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:05.919213    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:28:05.920571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:05.920541    2581 patch_prober.go:28] interesting pod/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 06 19:28:05.920719 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:05.920602    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 06 19:28:06.919600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:06.919562    2581 patch_prober.go:28] interesting pod/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 06 19:28:06.920101 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:06.919630    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 06 19:28:07.036822 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:07.036779    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928" event={"ID":"179bd0b3-5af5-4c5b-9fe9-dff976512c7f","Type":"ContainerStarted","Data":"0c1c52204e405c5ff69cd183fd18f1a7dda880273432309bce2d93bda1e5c7ac"}
Sep 06 19:28:07.051018 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:07.050958    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928" podStartSLOduration=2.25615664 podStartE2EDuration="5.050940942s" totalImagesPullingTime="2.794784302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:07.049670514 +0000 UTC m=+1015.330123323" watchObservedRunningTime="2026-09-06 19:28:07.050940942 +0000 UTC m=+1015.331393740"
Sep 06 19:28:07.920096 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:07.920035    2581 patch_prober.go:28] interesting pod/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 06 19:28:07.920497 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:07.920131    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 06 19:28:08.044543 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:08.044507    2581 generic.go:356] "Generic (PLEG): container finished" podID="179bd0b3-5af5-4c5b-9fe9-dff976512c7f" containerID="0c1c52204e405c5ff69cd183fd18f1a7dda880273432309bce2d93bda1e5c7ac" exitCode=0
Sep 06 19:28:08.044736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:08.044552    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928" event={"ID":"179bd0b3-5af5-4c5b-9fe9-dff976512c7f","Type":"ContainerDied","Data":"0c1c52204e405c5ff69cd183fd18f1a7dda880273432309bce2d93bda1e5c7ac"}
Sep 06 19:28:08.920277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:08.920232    2581 patch_prober.go:28] interesting pod/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 06 19:28:08.920785 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:08.920302    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 06 19:28:09.443426 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:09.443345    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"
Sep 06 19:28:09.572780 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:09.572743    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179bd0b3-5af5-4c5b-9fe9-dff976512c7f-maas-api-service-ca\") pod \"179bd0b3-5af5-4c5b-9fe9-dff976512c7f\" (UID: \"179bd0b3-5af5-4c5b-9fe9-dff976512c7f\") "
Sep 06 19:28:09.573238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:09.573212    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/179bd0b3-5af5-4c5b-9fe9-dff976512c7f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "179bd0b3-5af5-4c5b-9fe9-dff976512c7f" (UID: "179bd0b3-5af5-4c5b-9fe9-dff976512c7f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:28:09.673923 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:09.673881    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179bd0b3-5af5-4c5b-9fe9-dff976512c7f-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:09.768105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:09.767987    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:28:09.919372 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:09.919331    2581 patch_prober.go:28] interesting pod/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 06 19:28:09.920054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:09.919639    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 06 19:28:10.061056 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:10.060970    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928" event={"ID":"179bd0b3-5af5-4c5b-9fe9-dff976512c7f","Type":"ContainerDied","Data":"addca0cc49346579f8c9f7c0a2c2396b3620b33b049d81bee43ebe4e0f2c6024"}
Sep 06 19:28:10.061056 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:10.061031    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="addca0cc49346579f8c9f7c0a2c2396b3620b33b049d81bee43ebe4e0f2c6024"
Sep 06 19:28:10.061478 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:10.061124    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"
Sep 06 19:28:10.923661 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:10.923626    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:28:10.924714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:10.924695    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:28:18.118884 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:18.118787    2581 generic.go:356] "Generic (PLEG): container finished" podID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerID="4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36" exitCode=0
Sep 06 19:28:18.118884 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:18.118869    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" event={"ID":"2206bafb-7e52-48ec-9bb8-d661890a3a9f","Type":"ContainerDied","Data":"4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36"}
Sep 06 19:28:18.119367 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:18.119352    2581 scope.go:117] "RemoveContainer" containerID="4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36"
Sep 06 19:28:18.120558 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:18.120527    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:18.120687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:18.120642    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:18.120751 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:18.120694    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:18.492504 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:18.492463    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:19.029350 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.026294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:19.117958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.117821    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:19.131209 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.131162    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" event={"ID":"2206bafb-7e52-48ec-9bb8-d661890a3a9f","Type":"ContainerStarted","Data":"f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec"}
Sep 06 19:28:19.196430 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.196365    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:19.196632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.196501    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"]
Sep 06 19:28:19.208091 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.202928    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0281bd63-c2928"]
Sep 06 19:28:19.256458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.256427    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:19.325829 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.325734    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"]
Sep 06 19:28:19.326332 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.326294    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy" containerID="cri-o://11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8" gracePeriod=30
Sep 06 19:28:19.344047 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.344014    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:19.366734 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.366526    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:28:19.366734 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.366567    2581 patch_prober.go:28] interesting pod/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 06 19:28:19.366734 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.366618    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 06 19:28:19.777384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.777345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:19.869388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.869351    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:19.975876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:19.975843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:20.251003 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.250965    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"]
Sep 06 19:28:20.251666 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.251650    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="179bd0b3-5af5-4c5b-9fe9-dff976512c7f" containerName="revoke-keys"
Sep 06 19:28:20.251717 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.251672    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="179bd0b3-5af5-4c5b-9fe9-dff976512c7f" containerName="revoke-keys"
Sep 06 19:28:20.251799 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.251788    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="179bd0b3-5af5-4c5b-9fe9-dff976512c7f" containerName="revoke-keys"
Sep 06 19:28:20.255387 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.255367    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.258415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.258393    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-053eb1b8-gw-openshift-default-dockercfg-p54gk\""
Sep 06 19:28:20.268072 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.268038    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="179bd0b3-5af5-4c5b-9fe9-dff976512c7f" path="/var/lib/kubelet/pods/179bd0b3-5af5-4c5b-9fe9-dff976512c7f/volumes"
Sep 06 19:28:20.282984 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.282954    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"]
Sep 06 19:28:20.340734 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.340706    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:20.366449 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.366416    2581 patch_prober.go:28] interesting pod/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 06 19:28:20.366654 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.366478    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 06 19:28:20.416707 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.416651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-certs\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.416924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.416737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-data\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.416924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.416781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq8hz\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-kube-api-access-vq8hz\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.416924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.416835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-podinfo\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.416924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.416873    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-credential-socket\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.416924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.416894    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-token\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.416924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.416919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-socket\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.417263 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.416945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30980ae3-11e5-4f64-8308-3e1d81aee525-istiod-ca-cert\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.417263 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.416990    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-envoy\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.517714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.517613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-certs\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.517714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.517701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-data\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.517957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.517742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq8hz\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-kube-api-access-vq8hz\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.517957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.517792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-podinfo\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.517957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.517829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-credential-socket\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.517957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.517849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-token\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.517957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.517877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-socket\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.517957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.517902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30980ae3-11e5-4f64-8308-3e1d81aee525-istiod-ca-cert\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.517957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.517947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-envoy\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.518352 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.518096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-certs\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.518352 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.518163    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-credential-socket\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.518352 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.518176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-data\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.518529 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.518446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-socket\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.518697 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.518676    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30980ae3-11e5-4f64-8308-3e1d81aee525-istiod-ca-cert\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.520820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.520585    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-envoy\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.520820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.520775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-podinfo\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.525485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.525460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq8hz\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-kube-api-access-vq8hz\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.525597 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.525578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-token\") pod \"e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") " pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.567331 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.567296    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:20.811129 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.810964    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:20.906907 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.906876    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"]
Sep 06 19:28:20.917307 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.917254    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:20.917449 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.917360    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:20.917449 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:20.917404    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:21.143787 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:21.143748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" event={"ID":"30980ae3-11e5-4f64-8308-3e1d81aee525","Type":"ContainerStarted","Data":"c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e"}
Sep 06 19:28:21.143961 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:21.143815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" event={"ID":"30980ae3-11e5-4f64-8308-3e1d81aee525","Type":"ContainerStarted","Data":"9408f7056852fb399eaf812904e157f417c557eb7885057efd22b1842897488d"}
Sep 06 19:28:21.165314 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:21.165243    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" podStartSLOduration=1.165225674 podStartE2EDuration="1.165225674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:21.161327786 +0000 UTC m=+1029.441780580" watchObservedRunningTime="2026-09-06 19:28:21.165225674 +0000 UTC m=+1029.445678467"
Sep 06 19:28:21.366135 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:21.366100    2581 patch_prober.go:28] interesting pod/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 06 19:28:21.366624 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:21.366160    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 06 19:28:21.568223 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:21.568123    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:21.573926 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:21.573899    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:22.149633 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:22.149599    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:22.150675 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:22.150653    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:22.366324 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:22.366287    2581 patch_prober.go:28] interesting pod/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 06 19:28:22.366777 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:22.366353    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 06 19:28:22.662348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:22.662311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:22.771729 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:22.771687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:22.871253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:22.871217    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:22.934199 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:22.934106    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:23.039522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:23.039484    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:23.367309 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:23.367212    2581 patch_prober.go:28] interesting pod/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 06 19:28:23.367309 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:23.367278    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 06 19:28:23.402580 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:23.402535    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:23.462213 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:23.462158    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:24.626252 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.626222    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:28:24.631368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.631333    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:28:24.631647 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.631627    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:28:24.632742 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.632720    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:28:24.768135 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768058    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2hfl\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-kube-api-access-s2hfl\") pod \"746e2bc6-528e-4d55-a445-c35121151ce5\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") "
Sep 06 19:28:24.768135 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768128    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-socket\") pod \"746e2bc6-528e-4d55-a445-c35121151ce5\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") "
Sep 06 19:28:24.768387 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768234    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-credential-socket\") pod \"746e2bc6-528e-4d55-a445-c35121151ce5\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") "
Sep 06 19:28:24.768387 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768302    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746e2bc6-528e-4d55-a445-c35121151ce5-istio-podinfo\") pod \"746e2bc6-528e-4d55-a445-c35121151ce5\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") "
Sep 06 19:28:24.768387 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768336    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-istio-token\") pod \"746e2bc6-528e-4d55-a445-c35121151ce5\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") "
Sep 06 19:28:24.768387 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768364    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-certs\") pod \"746e2bc6-528e-4d55-a445-c35121151ce5\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") "
Sep 06 19:28:24.768387 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768373    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "746e2bc6-528e-4d55-a445-c35121151ce5" (UID: "746e2bc6-528e-4d55-a445-c35121151ce5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:24.768667 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768466    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "746e2bc6-528e-4d55-a445-c35121151ce5" (UID: "746e2bc6-528e-4d55-a445-c35121151ce5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:24.768667 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768490    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746e2bc6-528e-4d55-a445-c35121151ce5-istiod-ca-cert\") pod \"746e2bc6-528e-4d55-a445-c35121151ce5\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") "
Sep 06 19:28:24.768667 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-envoy\") pod \"746e2bc6-528e-4d55-a445-c35121151ce5\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") "
Sep 06 19:28:24.768667 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768559    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-data\") pod \"746e2bc6-528e-4d55-a445-c35121151ce5\" (UID: \"746e2bc6-528e-4d55-a445-c35121151ce5\") "
Sep 06 19:28:24.768923 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768756    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/746e2bc6-528e-4d55-a445-c35121151ce5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "746e2bc6-528e-4d55-a445-c35121151ce5" (UID: "746e2bc6-528e-4d55-a445-c35121151ce5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:28:24.768923 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768843    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-data" (OuterVolumeSpecName: "istio-data") pod "746e2bc6-528e-4d55-a445-c35121151ce5" (UID: "746e2bc6-528e-4d55-a445-c35121151ce5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:24.769026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.768940    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "746e2bc6-528e-4d55-a445-c35121151ce5" (UID: "746e2bc6-528e-4d55-a445-c35121151ce5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:24.769126 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.769092    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/746e2bc6-528e-4d55-a445-c35121151ce5-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:24.769126 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.769115    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:24.769238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.769129    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:24.769238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.769143    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:24.769238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.769159    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:24.770764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.770738    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/746e2bc6-528e-4d55-a445-c35121151ce5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "746e2bc6-528e-4d55-a445-c35121151ce5" (UID: "746e2bc6-528e-4d55-a445-c35121151ce5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:28:24.771002 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.770984    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-istio-token" (OuterVolumeSpecName: "istio-token") pod "746e2bc6-528e-4d55-a445-c35121151ce5" (UID: "746e2bc6-528e-4d55-a445-c35121151ce5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:28:24.771529 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.771505    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "746e2bc6-528e-4d55-a445-c35121151ce5" (UID: "746e2bc6-528e-4d55-a445-c35121151ce5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:24.771584 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.771535    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-kube-api-access-s2hfl" (OuterVolumeSpecName: "kube-api-access-s2hfl") pod "746e2bc6-528e-4d55-a445-c35121151ce5" (UID: "746e2bc6-528e-4d55-a445-c35121151ce5"). InnerVolumeSpecName "kube-api-access-s2hfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:28:24.870373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.870280    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/746e2bc6-528e-4d55-a445-c35121151ce5-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:24.870373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.870314    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2hfl\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-kube-api-access-s2hfl\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:24.870373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.870324    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/746e2bc6-528e-4d55-a445-c35121151ce5-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:24.870373 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:24.870332    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746e2bc6-528e-4d55-a445-c35121151ce5-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:25.169856 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.169811    2581 generic.go:356] "Generic (PLEG): container finished" podID="746e2bc6-528e-4d55-a445-c35121151ce5" containerID="11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8" exitCode=0
Sep 06 19:28:25.171248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.171151    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" event={"ID":"746e2bc6-528e-4d55-a445-c35121151ce5","Type":"ContainerDied","Data":"11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8"}
Sep 06 19:28:25.171248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.171179    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"
Sep 06 19:28:25.171471 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.171260    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z" event={"ID":"746e2bc6-528e-4d55-a445-c35121151ce5","Type":"ContainerDied","Data":"daad61ed45bc5e87e7826d9cdef229a7f7e6afc89a97cf707dbc3684f4ce6df4"}
Sep 06 19:28:25.171471 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.171298    2581 scope.go:117] "RemoveContainer" containerID="11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8"
Sep 06 19:28:25.183777 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.183756    2581 scope.go:117] "RemoveContainer" containerID="d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020"
Sep 06 19:28:25.194471 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.194441    2581 scope.go:117] "RemoveContainer" containerID="11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8"
Sep 06 19:28:25.194882 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:28:25.194860    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8\": container with ID starting with 11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8 not found: ID does not exist" containerID="11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8"
Sep 06 19:28:25.194960 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.194892    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8"} err="failed to get container status \"11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8\": rpc error: code = NotFound desc = could not find container \"11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8\": container with ID starting with 11879ce63cf0d9f67f2480a4245843399f02b52c39e03310387b2ddff554b8e8 not found: ID does not exist"
Sep 06 19:28:25.194960 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.194909    2581 scope.go:117] "RemoveContainer" containerID="d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020"
Sep 06 19:28:25.195861 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:28:25.195823    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020\": container with ID starting with d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020 not found: ID does not exist" containerID="d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020"
Sep 06 19:28:25.196142 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.195859    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020"} err="failed to get container status \"d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020\": rpc error: code = NotFound desc = could not find container \"d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020\": container with ID starting with d6a74c765e763611e34ebad73e6be31619bf6d9146d0ade4bc974ccbd6825020 not found: ID does not exist"
Sep 06 19:28:25.197522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.197501    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"]
Sep 06 19:28:25.199731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:25.199713    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0281bd63-openshift-default-64f789b4-55l5z"]
Sep 06 19:28:26.269180 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:26.269143    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" path="/var/lib/kubelet/pods/746e2bc6-528e-4d55-a445-c35121151ce5/volumes"
Sep 06 19:28:43.277920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:43.277884    2581 generic.go:356] "Generic (PLEG): container finished" podID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerID="c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e" exitCode=0
Sep 06 19:28:43.278401 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:43.277951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" event={"ID":"30980ae3-11e5-4f64-8308-3e1d81aee525","Type":"ContainerDied","Data":"c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e"}
Sep 06 19:28:43.278450 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:43.278406    2581 scope.go:117] "RemoveContainer" containerID="c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e"
Sep 06 19:28:43.280027 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:43.279994    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:43.280137 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:43.280107    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:43.280188 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:43.280152    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:44.287154 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:44.287111    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" event={"ID":"30980ae3-11e5-4f64-8308-3e1d81aee525","Type":"ContainerStarted","Data":"3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82"}
Sep 06 19:28:44.568402 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:44.568306    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:44.568584 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:44.568562    2581 patch_prober.go:28] interesting pod/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 06 19:28:44.568645 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:44.568626    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 06 19:28:45.568217 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:45.568181    2581 patch_prober.go:28] interesting pod/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 06 19:28:45.568655 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:45.568238    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 06 19:28:46.567716 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:46.567677    2581 patch_prober.go:28] interesting pod/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 06 19:28:46.567919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:46.567735    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 06 19:28:47.543786 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.543750    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"]
Sep 06 19:28:47.544234 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.544218    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy"
Sep 06 19:28:47.544282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.544237    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy"
Sep 06 19:28:47.544282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.544252    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy"
Sep 06 19:28:47.544282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.544259    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy"
Sep 06 19:28:47.544382 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.544336    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy"
Sep 06 19:28:47.544382 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.544345    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="746e2bc6-528e-4d55-a445-c35121151ce5" containerName="istio-proxy"
Sep 06 19:28:47.547626 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.547608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"
Sep 06 19:28:47.550011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.549981    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:28:47.550397 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.550366    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:28:47.552207 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.552182    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"]
Sep 06 19:28:47.568181 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.568145    2581 patch_prober.go:28] interesting pod/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 06 19:28:47.568348 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.568214    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 06 19:28:47.646326 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.646282    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4949c55f-8427-44cd-aeec-91d8d2260507-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff\" (UID: \"4949c55f-8427-44cd-aeec-91d8d2260507\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"
Sep 06 19:28:47.747745 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.747713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4949c55f-8427-44cd-aeec-91d8d2260507-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff\" (UID: \"4949c55f-8427-44cd-aeec-91d8d2260507\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"
Sep 06 19:28:47.748381 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.748359    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4949c55f-8427-44cd-aeec-91d8d2260507-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff\" (UID: \"4949c55f-8427-44cd-aeec-91d8d2260507\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"
Sep 06 19:28:47.861502 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.861418    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"
Sep 06 19:28:47.912842 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:47.912808    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:48.151783 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:48.151759    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"]
Sep 06 19:28:48.154106 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:28:48.154057    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4949c55f_8427_44cd_aeec_91d8d2260507.slice/crio-6ae6991cd3f6c5d7a6baca4afb7ed193c16814609f94426e57e2197d6aae903d WatchSource:0}: Error finding container 6ae6991cd3f6c5d7a6baca4afb7ed193c16814609f94426e57e2197d6aae903d: Status 404 returned error can't find the container with id 6ae6991cd3f6c5d7a6baca4afb7ed193c16814609f94426e57e2197d6aae903d
Sep 06 19:28:48.321604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:48.321565    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff" event={"ID":"4949c55f-8427-44cd-aeec-91d8d2260507","Type":"ContainerStarted","Data":"6ae6991cd3f6c5d7a6baca4afb7ed193c16814609f94426e57e2197d6aae903d"}
Sep 06 19:28:48.568137 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:48.568052    2581 patch_prober.go:28] interesting pod/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 06 19:28:48.568518 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:48.568159    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 06 19:28:49.329781 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:49.329745    2581 generic.go:356] "Generic (PLEG): container finished" podID="4949c55f-8427-44cd-aeec-91d8d2260507" containerID="ebcf102662f33864e451aba4b638e51a4bc113928bccc3eb2cd30aaa71afaea1" exitCode=0
Sep 06 19:28:49.329961 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:49.329825    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff" event={"ID":"4949c55f-8427-44cd-aeec-91d8d2260507","Type":"ContainerDied","Data":"ebcf102662f33864e451aba4b638e51a4bc113928bccc3eb2cd30aaa71afaea1"}
Sep 06 19:28:49.757015 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:49.756980    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:49.757622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:49.757392    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:49.758196 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:49.758177    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:50.056905 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:50.056823    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:50.484694 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:50.484667    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"
Sep 06 19:28:50.577233 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:50.577142    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4949c55f-8427-44cd-aeec-91d8d2260507-maas-api-service-ca\") pod \"4949c55f-8427-44cd-aeec-91d8d2260507\" (UID: \"4949c55f-8427-44cd-aeec-91d8d2260507\") "
Sep 06 19:28:50.577516 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:50.577492    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4949c55f-8427-44cd-aeec-91d8d2260507-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4949c55f-8427-44cd-aeec-91d8d2260507" (UID: "4949c55f-8427-44cd-aeec-91d8d2260507"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:28:50.678548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:50.678511    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4949c55f-8427-44cd-aeec-91d8d2260507-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:51.342596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:51.342560    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"
Sep 06 19:28:51.343006 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:51.342561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff" event={"ID":"4949c55f-8427-44cd-aeec-91d8d2260507","Type":"ContainerDied","Data":"6ae6991cd3f6c5d7a6baca4afb7ed193c16814609f94426e57e2197d6aae903d"}
Sep 06 19:28:51.343006 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:51.342668    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ae6991cd3f6c5d7a6baca4afb7ed193c16814609f94426e57e2197d6aae903d"
Sep 06 19:28:54.184086 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.180601    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:54.273224 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.273190    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:54.359389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.359359    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"]
Sep 06 19:28:54.361611 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.361584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:54.362356 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.362337    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-053eb1b8-bcsff"]
Sep 06 19:28:54.425358 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.425315    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:54.509174 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.497584    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"]
Sep 06 19:28:54.510054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.509988    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy" containerID="cri-o://3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82" gracePeriod=30
Sep 06 19:28:54.511941 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.511915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:54.908277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:54.908234    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:55.162772 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.162693    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"]
Sep 06 19:28:55.163415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.163394    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4949c55f-8427-44cd-aeec-91d8d2260507" containerName="revoke-keys"
Sep 06 19:28:55.163470 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.163421    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4949c55f-8427-44cd-aeec-91d8d2260507" containerName="revoke-keys"
Sep 06 19:28:55.163590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.163574    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4949c55f-8427-44cd-aeec-91d8d2260507" containerName="revoke-keys"
Sep 06 19:28:55.167585 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.167502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.169746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.169722    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-69e43c3f-openshift-default-dockercfg-mffld\""
Sep 06 19:28:55.180380 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.180349    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"]
Sep 06 19:28:55.193010 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.192846    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:55.228369 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.228331    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-podinfo\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.228560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.228386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-socket\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.228560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.228416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5s5b\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-kube-api-access-d5s5b\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.228560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.228452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-credential-socket\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.228560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.228484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-data\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.228724 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.228574    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-certs\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.228724 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.228668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istiod-ca-cert\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.228800 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.228749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-envoy\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.228800 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.228772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-token\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.329756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.329717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-podinfo\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.329947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.329787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-socket\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.329947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.329818    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5s5b\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-kube-api-access-d5s5b\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.329947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.329854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-credential-socket\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.329947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.329894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-data\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.329947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.329944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-certs\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.330287 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.329995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istiod-ca-cert\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.330287 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.330135    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-envoy\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.330287 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.330169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-token\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.330758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.330726    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-data\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.330880 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.330853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-certs\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.331046 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.331025    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-credential-socket\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.331461 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.331443    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-socket\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.331622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.331544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istiod-ca-cert\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.333947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.333926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-podinfo\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.334740 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.334720    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-envoy\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.337748 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.337724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-token\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.338255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.338236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5s5b\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-kube-api-access-d5s5b\") pod \"e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") " pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.481769 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.481682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:55.502655 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.502469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:55.767678 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.767582    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"]
Sep 06 19:28:55.770585 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:28:55.770551    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70855423_d9ac_4e8d_9e45_d54f76c86ba3.slice/crio-640df271518ef15d547b971fca29175ffaa9ead97165ace21b77f73135c3ba5e WatchSource:0}: Error finding container 640df271518ef15d547b971fca29175ffaa9ead97165ace21b77f73135c3ba5e: Status 404 returned error can't find the container with id 640df271518ef15d547b971fca29175ffaa9ead97165ace21b77f73135c3ba5e
Sep 06 19:28:55.773200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.773164    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:55.773318 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.773253    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:55.773386 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:55.773328    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:28:56.266540 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:56.266504    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4949c55f-8427-44cd-aeec-91d8d2260507" path="/var/lib/kubelet/pods/4949c55f-8427-44cd-aeec-91d8d2260507/volumes"
Sep 06 19:28:56.371243 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:56.371202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" event={"ID":"70855423-d9ac-4e8d-9e45-d54f76c86ba3","Type":"ContainerStarted","Data":"cde37de279914b5d94a78abed17a20dd2e089486c7af380f39e9ea2851baf21c"}
Sep 06 19:28:56.371447 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:56.371260    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" event={"ID":"70855423-d9ac-4e8d-9e45-d54f76c86ba3","Type":"ContainerStarted","Data":"640df271518ef15d547b971fca29175ffaa9ead97165ace21b77f73135c3ba5e"}
Sep 06 19:28:56.391436 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:56.391380    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" podStartSLOduration=1.39136218 podStartE2EDuration="1.39136218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:56.388372932 +0000 UTC m=+1064.668825725" watchObservedRunningTime="2026-09-06 19:28:56.39136218 +0000 UTC m=+1064.671814970"
Sep 06 19:28:56.482156 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:56.482118    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:56.487544 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:56.487514    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:57.380846 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:57.380815    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:57.383129 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:57.383102    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:28:57.699048 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:57.699012    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:57.793255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:57.793219    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:58.980257 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:58.980229    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:59.081571 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.081537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:59.520975 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.520942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:59.619633 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.619597    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:28:59.794579 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.794555    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:28:59.876820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.876785    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-data\") pod \"30980ae3-11e5-4f64-8308-3e1d81aee525\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") "
Sep 06 19:28:59.876820 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.876824    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30980ae3-11e5-4f64-8308-3e1d81aee525-istiod-ca-cert\") pod \"30980ae3-11e5-4f64-8308-3e1d81aee525\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") "
Sep 06 19:28:59.877052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.876885    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-envoy\") pod \"30980ae3-11e5-4f64-8308-3e1d81aee525\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") "
Sep 06 19:28:59.877052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.876910    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-credential-socket\") pod \"30980ae3-11e5-4f64-8308-3e1d81aee525\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") "
Sep 06 19:28:59.877052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.876940    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-podinfo\") pod \"30980ae3-11e5-4f64-8308-3e1d81aee525\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") "
Sep 06 19:28:59.877052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.876999    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq8hz\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-kube-api-access-vq8hz\") pod \"30980ae3-11e5-4f64-8308-3e1d81aee525\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") "
Sep 06 19:28:59.877052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877029    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-certs\") pod \"30980ae3-11e5-4f64-8308-3e1d81aee525\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") "
Sep 06 19:28:59.877361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877149    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-socket\") pod \"30980ae3-11e5-4f64-8308-3e1d81aee525\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") "
Sep 06 19:28:59.877361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877159    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-data" (OuterVolumeSpecName: "istio-data") pod "30980ae3-11e5-4f64-8308-3e1d81aee525" (UID: "30980ae3-11e5-4f64-8308-3e1d81aee525"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:59.877361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877195    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-token\") pod \"30980ae3-11e5-4f64-8308-3e1d81aee525\" (UID: \"30980ae3-11e5-4f64-8308-3e1d81aee525\") "
Sep 06 19:28:59.877361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877248    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30980ae3-11e5-4f64-8308-3e1d81aee525" (UID: "30980ae3-11e5-4f64-8308-3e1d81aee525"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:59.877577 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877409    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30980ae3-11e5-4f64-8308-3e1d81aee525" (UID: "30980ae3-11e5-4f64-8308-3e1d81aee525"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:59.877577 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30980ae3-11e5-4f64-8308-3e1d81aee525" (UID: "30980ae3-11e5-4f64-8308-3e1d81aee525"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:59.877577 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877438    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30980ae3-11e5-4f64-8308-3e1d81aee525-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30980ae3-11e5-4f64-8308-3e1d81aee525" (UID: "30980ae3-11e5-4f64-8308-3e1d81aee525"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:28:59.877913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877593    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:59.877913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877609    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30980ae3-11e5-4f64-8308-3e1d81aee525-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:59.877913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877622    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:59.877913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877631    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:59.877913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.877639    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:59.879291 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.879257    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30980ae3-11e5-4f64-8308-3e1d81aee525" (UID: "30980ae3-11e5-4f64-8308-3e1d81aee525"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:28:59.879413 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.879393    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-kube-api-access-vq8hz" (OuterVolumeSpecName: "kube-api-access-vq8hz") pod "30980ae3-11e5-4f64-8308-3e1d81aee525" (UID: "30980ae3-11e5-4f64-8308-3e1d81aee525"). InnerVolumeSpecName "kube-api-access-vq8hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:28:59.879475 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.879451    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30980ae3-11e5-4f64-8308-3e1d81aee525" (UID: "30980ae3-11e5-4f64-8308-3e1d81aee525"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:28:59.879557 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.879540    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-token" (OuterVolumeSpecName: "istio-token") pod "30980ae3-11e5-4f64-8308-3e1d81aee525" (UID: "30980ae3-11e5-4f64-8308-3e1d81aee525"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:28:59.979271 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.979223    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:59.979271 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.979270    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:59.979271 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.979279    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq8hz\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-kube-api-access-vq8hz\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:28:59.979271 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:28:59.979289    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30980ae3-11e5-4f64-8308-3e1d81aee525-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:00.399347 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.399312    2581 generic.go:356] "Generic (PLEG): container finished" podID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerID="3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82" exitCode=0
Sep 06 19:29:00.399797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.399371    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"
Sep 06 19:29:00.399797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.399397    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" event={"ID":"30980ae3-11e5-4f64-8308-3e1d81aee525","Type":"ContainerDied","Data":"3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82"}
Sep 06 19:29:00.399797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.399465    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk" event={"ID":"30980ae3-11e5-4f64-8308-3e1d81aee525","Type":"ContainerDied","Data":"9408f7056852fb399eaf812904e157f417c557eb7885057efd22b1842897488d"}
Sep 06 19:29:00.399797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.399500    2581 scope.go:117] "RemoveContainer" containerID="3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82"
Sep 06 19:29:00.409153 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.409127    2581 scope.go:117] "RemoveContainer" containerID="c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e"
Sep 06 19:29:00.419052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.418953    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"]
Sep 06 19:29:00.419198 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.419180    2581 scope.go:117] "RemoveContainer" containerID="3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82"
Sep 06 19:29:00.419519 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:29:00.419500    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82\": container with ID starting with 3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82 not found: ID does not exist" containerID="3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82"
Sep 06 19:29:00.419567 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.419527    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82"} err="failed to get container status \"3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82\": rpc error: code = NotFound desc = could not find container \"3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82\": container with ID starting with 3a5d3f5834b7b9959e792e7fe1f80510eb3bf76365a32b2124c57f939bffcf82 not found: ID does not exist"
Sep 06 19:29:00.419567 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.419544    2581 scope.go:117] "RemoveContainer" containerID="c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e"
Sep 06 19:29:00.420414 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:29:00.420041    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e\": container with ID starting with c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e not found: ID does not exist" containerID="c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e"
Sep 06 19:29:00.420414 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.420103    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e"} err="failed to get container status \"c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e\": rpc error: code = NotFound desc = could not find container \"c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e\": container with ID starting with c8fbc979a8d115e102ad26fc33ace3628065757d722f24190cb749ae287c4b7e not found: ID does not exist"
Sep 06 19:29:00.422620 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:00.422600    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-053eb1b8-gw-openshift-default-b68bc7f5b-wlzvk"]
Sep 06 19:29:02.267942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:02.267902    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" path="/var/lib/kubelet/pods/30980ae3-11e5-4f64-8308-3e1d81aee525/volumes"
Sep 06 19:29:13.422271 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:13.422230    2581 patch_prober.go:28] interesting pod/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 19:29:13.422794 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:13.422314    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 19:29:14.502262 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:14.502227    2581 generic.go:356] "Generic (PLEG): container finished" podID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerID="cde37de279914b5d94a78abed17a20dd2e089486c7af380f39e9ea2851baf21c" exitCode=0
Sep 06 19:29:14.502712 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:14.502304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" event={"ID":"70855423-d9ac-4e8d-9e45-d54f76c86ba3","Type":"ContainerDied","Data":"cde37de279914b5d94a78abed17a20dd2e089486c7af380f39e9ea2851baf21c"}
Sep 06 19:29:14.502799 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:14.502785    2581 scope.go:117] "RemoveContainer" containerID="cde37de279914b5d94a78abed17a20dd2e089486c7af380f39e9ea2851baf21c"
Sep 06 19:29:14.503926 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:14.503901    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:29:14.503994 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:14.503966    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:29:14.504032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:14.503996    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:29:15.515467 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:15.515436    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" event={"ID":"70855423-d9ac-4e8d-9e45-d54f76c86ba3","Type":"ContainerStarted","Data":"5f7bf9a592de55df9183b7c879c32d923b8d3404cf03f6a8e9054bd35c415040"}
Sep 06 19:29:16.482111 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:16.482054    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:29:16.482396 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:16.482291    2581 patch_prober.go:28] interesting pod/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 06 19:29:16.482396 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:16.482353    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 06 19:29:17.482876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:17.482830    2581 patch_prober.go:28] interesting pod/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 06 19:29:17.483400 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:17.482894    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 06 19:29:18.482450 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:18.482409    2581 patch_prober.go:28] interesting pod/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 06 19:29:18.482638 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:18.482473    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 06 19:29:19.482475 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:19.482437    2581 patch_prober.go:28] interesting pod/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 06 19:29:19.482880 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:19.482495    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 06 19:29:20.482360 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:20.482318    2581 patch_prober.go:28] interesting pod/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 06 19:29:20.482550 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:20.482376    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 06 19:29:21.486415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:21.486369    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:29:21.486913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:21.486890    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:29:21.487825 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:21.487805    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:29:23.346958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.346919    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"]
Sep 06 19:29:23.347363 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.347355    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy"
Sep 06 19:29:23.347407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.347367    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy"
Sep 06 19:29:23.347456 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.347445    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy"
Sep 06 19:29:23.350848 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.350830    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"
Sep 06 19:29:23.353012 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.352983    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:29:23.353183 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.353011    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:29:23.356972 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.356947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"]
Sep 06 19:29:23.439818 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.439776    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c4657f-5f27-4957-8363-6fb0bdf5d6e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q\" (UID: \"c0c4657f-5f27-4957-8363-6fb0bdf5d6e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"
Sep 06 19:29:23.540620 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.540574    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c4657f-5f27-4957-8363-6fb0bdf5d6e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q\" (UID: \"c0c4657f-5f27-4957-8363-6fb0bdf5d6e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"
Sep 06 19:29:23.541317 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.541294    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c4657f-5f27-4957-8363-6fb0bdf5d6e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q\" (UID: \"c0c4657f-5f27-4957-8363-6fb0bdf5d6e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"
Sep 06 19:29:23.664731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.664693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"
Sep 06 19:29:23.936178 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:23.936150    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"]
Sep 06 19:29:23.938041 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:29:23.938013    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c4657f_5f27_4957_8363_6fb0bdf5d6e9.slice/crio-3a55df02d146a28c9eab3b61913dc380532321afea875c155434f3c3f845bb54 WatchSource:0}: Error finding container 3a55df02d146a28c9eab3b61913dc380532321afea875c155434f3c3f845bb54: Status 404 returned error can't find the container with id 3a55df02d146a28c9eab3b61913dc380532321afea875c155434f3c3f845bb54
Sep 06 19:29:24.572944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:24.572848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q" event={"ID":"c0c4657f-5f27-4957-8363-6fb0bdf5d6e9","Type":"ContainerStarted","Data":"645de9b37df7850c9761eaf81eb1be6987b0d88b5f3fb379cc6260651453e32f"}
Sep 06 19:29:24.573322 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:24.572946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q" event={"ID":"c0c4657f-5f27-4957-8363-6fb0bdf5d6e9","Type":"ContainerStarted","Data":"3a55df02d146a28c9eab3b61913dc380532321afea875c155434f3c3f845bb54"}
Sep 06 19:29:24.589353 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:24.589309    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q" podStartSLOduration=1.589292712 podStartE2EDuration="1.589292712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:24.586921245 +0000 UTC m=+1092.867374039" watchObservedRunningTime="2026-09-06 19:29:24.589292712 +0000 UTC m=+1092.869745505"
Sep 06 19:29:25.579635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:25.579598    2581 generic.go:356] "Generic (PLEG): container finished" podID="c0c4657f-5f27-4957-8363-6fb0bdf5d6e9" containerID="645de9b37df7850c9761eaf81eb1be6987b0d88b5f3fb379cc6260651453e32f" exitCode=0
Sep 06 19:29:25.580088 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:25.579690    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q" event={"ID":"c0c4657f-5f27-4957-8363-6fb0bdf5d6e9","Type":"ContainerDied","Data":"645de9b37df7850c9761eaf81eb1be6987b0d88b5f3fb379cc6260651453e32f"}
Sep 06 19:29:26.735918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:26.735882    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"
Sep 06 19:29:26.772094 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:26.772048    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c4657f-5f27-4957-8363-6fb0bdf5d6e9-maas-api-service-ca\") pod \"c0c4657f-5f27-4957-8363-6fb0bdf5d6e9\" (UID: \"c0c4657f-5f27-4957-8363-6fb0bdf5d6e9\") "
Sep 06 19:29:26.772488 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:26.772454    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0c4657f-5f27-4957-8363-6fb0bdf5d6e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0c4657f-5f27-4957-8363-6fb0bdf5d6e9" (UID: "c0c4657f-5f27-4957-8363-6fb0bdf5d6e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:29:26.873421 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:26.873328    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c4657f-5f27-4957-8363-6fb0bdf5d6e9-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:27.592306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:27.592281    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"
Sep 06 19:29:27.592481 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:27.592330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q" event={"ID":"c0c4657f-5f27-4957-8363-6fb0bdf5d6e9","Type":"ContainerDied","Data":"3a55df02d146a28c9eab3b61913dc380532321afea875c155434f3c3f845bb54"}
Sep 06 19:29:27.592481 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:27.592381    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a55df02d146a28c9eab3b61913dc380532321afea875c155434f3c3f845bb54"
Sep 06 19:29:29.242102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:29.242047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:29.302317 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:29.302289    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:29.358998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:29.358962    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:29.408296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:29.408258    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"]
Sep 06 19:29:29.410392 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:29.410353    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69e43c3f-6st6q"]
Sep 06 19:29:29.413561 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:29.413536    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:29.466207 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:29.466170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:30.266569 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:30.266538    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0c4657f-5f27-4957-8363-6fb0bdf5d6e9" path="/var/lib/kubelet/pods/c0c4657f-5f27-4957-8363-6fb0bdf5d6e9/volumes"
Sep 06 19:29:30.440557 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:30.440524    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"]
Sep 06 19:29:30.440832 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:30.440805    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy" containerID="cri-o://5f7bf9a592de55df9183b7c879c32d923b8d3404cf03f6a8e9054bd35c415040" gracePeriod=30
Sep 06 19:29:30.462223 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:30.462194    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:30.867476 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:30.867443    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:31.128005 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.127911    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"]
Sep 06 19:29:31.128449 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.128432    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy"
Sep 06 19:29:31.128515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.128451    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy"
Sep 06 19:29:31.128515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.128469    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c4657f-5f27-4957-8363-6fb0bdf5d6e9" containerName="revoke-keys"
Sep 06 19:29:31.128515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.128478    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c4657f-5f27-4957-8363-6fb0bdf5d6e9" containerName="revoke-keys"
Sep 06 19:29:31.128626 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.128553    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0c4657f-5f27-4957-8363-6fb0bdf5d6e9" containerName="revoke-keys"
Sep 06 19:29:31.128626 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.128567    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="30980ae3-11e5-4f64-8308-3e1d81aee525" containerName="istio-proxy"
Sep 06 19:29:31.132261 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.132239    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.137326 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.137300    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-119d0999-openshift-default-dockercfg-xgtj8\""
Sep 06 19:29:31.146491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.146463    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"]
Sep 06 19:29:31.172956 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.172914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:31.225234 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.225189    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-podinfo\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.225459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.225252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-certs\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.225459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.225290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-envoy\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.225459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.225334    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc86x\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-kube-api-access-zc86x\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.225459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.225365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-socket\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.225459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.225388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istiod-ca-cert\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.225760 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.225460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-credential-socket\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.225760 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.225559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-data\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.225760 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.225583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-token\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.326786 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.326740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-data\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.326786 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.326785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-token\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.326837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-podinfo\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.326861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-certs\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.326880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-envoy\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.326917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc86x\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-kube-api-access-zc86x\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.326948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-socket\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.327192    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istiod-ca-cert\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.327297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-credential-socket\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.327300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-data\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.327358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-socket\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.327358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-certs\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.327851 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.327578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-credential-socket\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.328235 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.328210    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istiod-ca-cert\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.329517 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.329489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-envoy\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.329637 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.329538    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-podinfo\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.338632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.338605    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-token\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.338780 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.338766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc86x\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-kube-api-access-zc86x\") pod \"e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") " pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.443341 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.443295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:31.530406 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.530361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:31.719573 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.719543    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"]
Sep 06 19:29:31.721216 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:29:31.721185    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode91a1428_cf0a_4149_84d5_2ec93207c5bb.slice/crio-77108358edc60684d57f2a72419c1fdb705c06fd4bab3fe9468183a253e12b87 WatchSource:0}: Error finding container 77108358edc60684d57f2a72419c1fdb705c06fd4bab3fe9468183a253e12b87: Status 404 returned error can't find the container with id 77108358edc60684d57f2a72419c1fdb705c06fd4bab3fe9468183a253e12b87
Sep 06 19:29:31.723533 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.723499    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:29:31.723611 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.723576    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:29:31.723611 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:31.723605    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:29:32.623484 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:32.623451    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" event={"ID":"e91a1428-cf0a-4149-84d5-2ec93207c5bb","Type":"ContainerStarted","Data":"b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d"}
Sep 06 19:29:32.623982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:32.623521    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" event={"ID":"e91a1428-cf0a-4149-84d5-2ec93207c5bb","Type":"ContainerStarted","Data":"77108358edc60684d57f2a72419c1fdb705c06fd4bab3fe9468183a253e12b87"}
Sep 06 19:29:32.642877 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:32.642822    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" podStartSLOduration=1.642803236 podStartE2EDuration="1.642803236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:32.640031634 +0000 UTC m=+1100.920484428" watchObservedRunningTime="2026-09-06 19:29:32.642803236 +0000 UTC m=+1100.923256030"
Sep 06 19:29:33.444265 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:33.444231    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:33.449915 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:33.449887    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:33.628935 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:33.628898    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:33.630129 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:33.630113    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:33.968430 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:33.968397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:34.059524 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:34.059488    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:35.206652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.206614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:35.307754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.307709    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:35.512079 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.512043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:35.657189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.657148    2581 generic.go:356] "Generic (PLEG): container finished" podID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerID="5f7bf9a592de55df9183b7c879c32d923b8d3404cf03f6a8e9054bd35c415040" exitCode=0
Sep 06 19:29:35.657444 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.657238    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" event={"ID":"70855423-d9ac-4e8d-9e45-d54f76c86ba3","Type":"ContainerDied","Data":"5f7bf9a592de55df9183b7c879c32d923b8d3404cf03f6a8e9054bd35c415040"}
Sep 06 19:29:35.657528 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.657412    2581 scope.go:117] "RemoveContainer" containerID="cde37de279914b5d94a78abed17a20dd2e089486c7af380f39e9ea2851baf21c"
Sep 06 19:29:35.671195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.671164    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:35.709899 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.709865    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:29:35.736996 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.736907    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:29:35.776183 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776149    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5s5b\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-kube-api-access-d5s5b\") pod \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") "
Sep 06 19:29:35.776368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776230    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-podinfo\") pod \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") "
Sep 06 19:29:35.776368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776268    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istiod-ca-cert\") pod \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") "
Sep 06 19:29:35.776368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776304    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-credential-socket\") pod \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") "
Sep 06 19:29:35.776532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776421    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-certs\") pod \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") "
Sep 06 19:29:35.776532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776467    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-envoy\") pod \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") "
Sep 06 19:29:35.776635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776550    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-socket\") pod \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") "
Sep 06 19:29:35.776635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776579    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-data\") pod \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") "
Sep 06 19:29:35.776635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776607    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-token\") pod \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\" (UID: \"70855423-d9ac-4e8d-9e45-d54f76c86ba3\") "
Sep 06 19:29:35.776635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776616    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70855423-d9ac-4e8d-9e45-d54f76c86ba3" (UID: "70855423-d9ac-4e8d-9e45-d54f76c86ba3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:29:35.776860 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776839    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70855423-d9ac-4e8d-9e45-d54f76c86ba3" (UID: "70855423-d9ac-4e8d-9e45-d54f76c86ba3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:29:35.776860 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.776764    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70855423-d9ac-4e8d-9e45-d54f76c86ba3" (UID: "70855423-d9ac-4e8d-9e45-d54f76c86ba3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:29:35.777212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.777188    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:35.777301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.777216    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:35.777301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.777230    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:35.777423 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.777407    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70855423-d9ac-4e8d-9e45-d54f76c86ba3" (UID: "70855423-d9ac-4e8d-9e45-d54f76c86ba3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:29:35.777488 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.777430    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-data" (OuterVolumeSpecName: "istio-data") pod "70855423-d9ac-4e8d-9e45-d54f76c86ba3" (UID: "70855423-d9ac-4e8d-9e45-d54f76c86ba3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:29:35.779261 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.779233    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70855423-d9ac-4e8d-9e45-d54f76c86ba3" (UID: "70855423-d9ac-4e8d-9e45-d54f76c86ba3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:29:35.779625 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.779590    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-kube-api-access-d5s5b" (OuterVolumeSpecName: "kube-api-access-d5s5b") pod "70855423-d9ac-4e8d-9e45-d54f76c86ba3" (UID: "70855423-d9ac-4e8d-9e45-d54f76c86ba3"). InnerVolumeSpecName "kube-api-access-d5s5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:29:35.779942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.779912    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70855423-d9ac-4e8d-9e45-d54f76c86ba3" (UID: "70855423-d9ac-4e8d-9e45-d54f76c86ba3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:29:35.780171 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.780150    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-token" (OuterVolumeSpecName: "istio-token") pod "70855423-d9ac-4e8d-9e45-d54f76c86ba3" (UID: "70855423-d9ac-4e8d-9e45-d54f76c86ba3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:29:35.890727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.881559    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:35.890727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.881615    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:35.890727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.881634    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:35.890727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.881650    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:35.890727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.881682    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5s5b\" (UniqueName: \"kubernetes.io/projected/70855423-d9ac-4e8d-9e45-d54f76c86ba3-kube-api-access-d5s5b\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:35.890727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:35.881709    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70855423-d9ac-4e8d-9e45-d54f76c86ba3-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:29:36.665264 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:36.665227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq" event={"ID":"70855423-d9ac-4e8d-9e45-d54f76c86ba3","Type":"ContainerDied","Data":"640df271518ef15d547b971fca29175ffaa9ead97165ace21b77f73135c3ba5e"}
Sep 06 19:29:36.665677 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:36.665294    2581 scope.go:117] "RemoveContainer" containerID="5f7bf9a592de55df9183b7c879c32d923b8d3404cf03f6a8e9054bd35c415040"
Sep 06 19:29:36.665677 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:36.665233    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"
Sep 06 19:29:36.682418 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:36.682379    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"]
Sep 06 19:29:36.684593 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:36.684571    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-69e43c3f-openshift-default-5699b96f54-nz4tq"]
Sep 06 19:29:38.267669 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:38.267636    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" path="/var/lib/kubelet/pods/70855423-d9ac-4e8d-9e45-d54f76c86ba3/volumes"
Sep 06 19:29:54.767790 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:54.767753    2581 generic.go:356] "Generic (PLEG): container finished" podID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerID="b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d" exitCode=0
Sep 06 19:29:54.768295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:54.767808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" event={"ID":"e91a1428-cf0a-4149-84d5-2ec93207c5bb","Type":"ContainerDied","Data":"b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d"}
Sep 06 19:29:54.768295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:54.768244    2581 scope.go:117] "RemoveContainer" containerID="b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d"
Sep 06 19:29:54.769509 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:54.769474    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:29:54.769600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:54.769583    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:29:54.769644 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:54.769631    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:29:55.775638 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:55.775599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" event={"ID":"e91a1428-cf0a-4149-84d5-2ec93207c5bb","Type":"ContainerStarted","Data":"f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec"}
Sep 06 19:29:56.443811 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:56.443752    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:29:56.444056 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:56.444030    2581 patch_prober.go:28] interesting pod/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 06 19:29:56.444268 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:56.444239    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 06 19:29:57.444637 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:57.444593    2581 patch_prober.go:28] interesting pod/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 06 19:29:57.445149 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:57.444656    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 06 19:29:58.443720 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.443682    2581 patch_prober.go:28] interesting pod/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 06 19:29:58.443899 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.443761    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 06 19:29:58.762366 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.762264    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"]
Sep 06 19:29:58.762913 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.762892    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy"
Sep 06 19:29:58.763002 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.762916    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy"
Sep 06 19:29:58.763095 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.763041    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy"
Sep 06 19:29:58.763095 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.763056    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy"
Sep 06 19:29:58.766621 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.766599    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"
Sep 06 19:29:58.769298 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.769273    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:29:58.769498 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.769285    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:29:58.771813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.771787    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"]
Sep 06 19:29:58.833056 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.833019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afdfb772-3a6b-4d81-b691-0dcf6f13f004-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-119d0999-8crsw\" (UID: \"afdfb772-3a6b-4d81-b691-0dcf6f13f004\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"
Sep 06 19:29:58.933856 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.933819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afdfb772-3a6b-4d81-b691-0dcf6f13f004-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-119d0999-8crsw\" (UID: \"afdfb772-3a6b-4d81-b691-0dcf6f13f004\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"
Sep 06 19:29:58.934524 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:58.934504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afdfb772-3a6b-4d81-b691-0dcf6f13f004-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-119d0999-8crsw\" (UID: \"afdfb772-3a6b-4d81-b691-0dcf6f13f004\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"
Sep 06 19:29:59.079964 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:59.079884    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"
Sep 06 19:29:59.374174 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:59.374147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"]
Sep 06 19:29:59.375973 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:29:59.375932    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafdfb772_3a6b_4d81_b691_0dcf6f13f004.slice/crio-fc406e292919b682a7eafbf83dae321218e387a2c22276eb71deb0de7b385286 WatchSource:0}: Error finding container fc406e292919b682a7eafbf83dae321218e387a2c22276eb71deb0de7b385286: Status 404 returned error can't find the container with id fc406e292919b682a7eafbf83dae321218e387a2c22276eb71deb0de7b385286
Sep 06 19:29:59.444162 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:59.444130    2581 patch_prober.go:28] interesting pod/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 06 19:29:59.444304 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:59.444192    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 06 19:29:59.823681 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:59.823650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw" event={"ID":"afdfb772-3a6b-4d81-b691-0dcf6f13f004","Type":"ContainerStarted","Data":"73ca7b1c29c0538e6ca5d758cd0a2b42cc3b20be406fb84c1b6a8c10eb0b9a43"}
Sep 06 19:29:59.824053 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:59.823699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw" event={"ID":"afdfb772-3a6b-4d81-b691-0dcf6f13f004","Type":"ContainerStarted","Data":"fc406e292919b682a7eafbf83dae321218e387a2c22276eb71deb0de7b385286"}
Sep 06 19:29:59.839305 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:29:59.839261    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw" podStartSLOduration=1.8392426309999999 podStartE2EDuration="1.839242631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:59.835964779 +0000 UTC m=+1128.116417573" watchObservedRunningTime="2026-09-06 19:29:59.839242631 +0000 UTC m=+1128.119695426"
Sep 06 19:30:00.131696 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.131663    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p"]
Sep 06 19:30:00.132560 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.132535    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy"
Sep 06 19:30:00.132740 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.132727    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="70855423-d9ac-4e8d-9e45-d54f76c86ba3" containerName="istio-proxy"
Sep 06 19:30:00.136527 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.136505    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb"]
Sep 06 19:30:00.137052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.136623    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p"
Sep 06 19:30:00.140798 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.140698    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc"]
Sep 06 19:30:00.140876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.140817    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb"
Sep 06 19:30:00.148368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.148250    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p"]
Sep 06 19:30:00.148544 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.148530    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb"]
Sep 06 19:30:00.148652 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.148343    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc"
Sep 06 19:30:00.152134 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.152113    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc"]
Sep 06 19:30:00.153116 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.152925    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p"
Sep 06 19:30:00.162865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.162843    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb"
Sep 06 19:30:00.218703 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.218670    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc"
Sep 06 19:30:00.444467 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.444429    2581 patch_prober.go:28] interesting pod/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 06 19:30:00.444668 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.444504    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 06 19:30:00.477227 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.476953    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p"]
Sep 06 19:30:00.479852 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.479823    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb"]
Sep 06 19:30:00.481243 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:30:00.481145    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077e96db_14a7_4f7d_8517_34414ea741c9.slice/crio-d0a6025e7b84a1cae50a06dafdeedcec2f748462726b4a535ef968eb4cb4c8b5 WatchSource:0}: Error finding container d0a6025e7b84a1cae50a06dafdeedcec2f748462726b4a535ef968eb4cb4c8b5: Status 404 returned error can't find the container with id d0a6025e7b84a1cae50a06dafdeedcec2f748462726b4a535ef968eb4cb4c8b5
Sep 06 19:30:00.481914 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:30:00.481892    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee859d70_b16a_49d3_83eb_1b0d5f50e892.slice/crio-c0d2303cb4080b9e75063b1aea931b821f32df64efa5078ef3a4a69d4bf1affc WatchSource:0}: Error finding container c0d2303cb4080b9e75063b1aea931b821f32df64efa5078ef3a4a69d4bf1affc: Status 404 returned error can't find the container with id c0d2303cb4080b9e75063b1aea931b821f32df64efa5078ef3a4a69d4bf1affc
Sep 06 19:30:00.545391 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.545365    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc"]
Sep 06 19:30:00.546474 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:30:00.546449    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa0a1aa2_13f9_4f7f_bff5_5020ddf5e816.slice/crio-8870a68e68a576055c9726c2677254557b8d76c89283217c02c55d858152e0e3 WatchSource:0}: Error finding container 8870a68e68a576055c9726c2677254557b8d76c89283217c02c55d858152e0e3: Status 404 returned error can't find the container with id 8870a68e68a576055c9726c2677254557b8d76c89283217c02c55d858152e0e3
Sep 06 19:30:00.831509 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.831465    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc" event={"ID":"aa0a1aa2-13f9-4f7f-bff5-5020ddf5e816","Type":"ContainerStarted","Data":"8870a68e68a576055c9726c2677254557b8d76c89283217c02c55d858152e0e3"}
Sep 06 19:30:00.832972 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.832946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p" event={"ID":"077e96db-14a7-4f7d-8517-34414ea741c9","Type":"ContainerStarted","Data":"e404e7f0401020f8a7da926cfa358aa55875ae38b4a8b27a59868d7b10ff96c1"}
Sep 06 19:30:00.833093 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.832996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p" event={"ID":"077e96db-14a7-4f7d-8517-34414ea741c9","Type":"ContainerStarted","Data":"d0a6025e7b84a1cae50a06dafdeedcec2f748462726b4a535ef968eb4cb4c8b5"}
Sep 06 19:30:00.834138 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.834120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb" event={"ID":"ee859d70-b16a-49d3-83eb-1b0d5f50e892","Type":"ContainerStarted","Data":"c0d2303cb4080b9e75063b1aea931b821f32df64efa5078ef3a4a69d4bf1affc"}
Sep 06 19:30:00.835594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.835516    2581 generic.go:356] "Generic (PLEG): container finished" podID="afdfb772-3a6b-4d81-b691-0dcf6f13f004" containerID="73ca7b1c29c0538e6ca5d758cd0a2b42cc3b20be406fb84c1b6a8c10eb0b9a43" exitCode=0
Sep 06 19:30:00.835594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.835561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw" event={"ID":"afdfb772-3a6b-4d81-b691-0dcf6f13f004","Type":"ContainerDied","Data":"73ca7b1c29c0538e6ca5d758cd0a2b42cc3b20be406fb84c1b6a8c10eb0b9a43"}
Sep 06 19:30:00.850180 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:00.850135    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p" podStartSLOduration=0.850121699 podStartE2EDuration="850.121699ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:30:00.84823484 +0000 UTC m=+1129.128687633" watchObservedRunningTime="2026-09-06 19:30:00.850121699 +0000 UTC m=+1129.130574492"
Sep 06 19:30:01.444542 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:01.444508    2581 patch_prober.go:28] interesting pod/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 06 19:30:01.444727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:01.444565    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 06 19:30:01.844302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:01.844259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb" event={"ID":"ee859d70-b16a-49d3-83eb-1b0d5f50e892","Type":"ContainerStarted","Data":"3f205ed779876216870243b79de138e626cfd9c960d1a789385dbc4d6c98b6c3"}
Sep 06 19:30:01.846194 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:01.846161    2581 generic.go:356] "Generic (PLEG): container finished" podID="aa0a1aa2-13f9-4f7f-bff5-5020ddf5e816" containerID="40bc996764c31183887d8532527d31b1b8e5d257689d16f5527f8b0e228b6402" exitCode=0
Sep 06 19:30:01.846322 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:01.846243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc" event={"ID":"aa0a1aa2-13f9-4f7f-bff5-5020ddf5e816","Type":"ContainerDied","Data":"40bc996764c31183887d8532527d31b1b8e5d257689d16f5527f8b0e228b6402"}
Sep 06 19:30:01.860418 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:01.860355    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb" podStartSLOduration=1.860331586 podStartE2EDuration="1.860331586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:30:01.856467297 +0000 UTC m=+1130.136920092" watchObservedRunningTime="2026-09-06 19:30:01.860331586 +0000 UTC m=+1130.140784381"
Sep 06 19:30:02.028372 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.028344    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"
Sep 06 19:30:02.068940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.068901    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afdfb772-3a6b-4d81-b691-0dcf6f13f004-maas-api-service-ca\") pod \"afdfb772-3a6b-4d81-b691-0dcf6f13f004\" (UID: \"afdfb772-3a6b-4d81-b691-0dcf6f13f004\") "
Sep 06 19:30:02.069292 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.069266    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afdfb772-3a6b-4d81-b691-0dcf6f13f004-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "afdfb772-3a6b-4d81-b691-0dcf6f13f004" (UID: "afdfb772-3a6b-4d81-b691-0dcf6f13f004"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:30:02.170213 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.170177    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afdfb772-3a6b-4d81-b691-0dcf6f13f004-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:02.449051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.449011    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:30:02.449381 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.449357    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:30:02.450492 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.450460    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:30:02.854293 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.854256    2581 generic.go:356] "Generic (PLEG): container finished" podID="077e96db-14a7-4f7d-8517-34414ea741c9" containerID="e404e7f0401020f8a7da926cfa358aa55875ae38b4a8b27a59868d7b10ff96c1" exitCode=0
Sep 06 19:30:02.854746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.854336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p" event={"ID":"077e96db-14a7-4f7d-8517-34414ea741c9","Type":"ContainerDied","Data":"e404e7f0401020f8a7da926cfa358aa55875ae38b4a8b27a59868d7b10ff96c1"}
Sep 06 19:30:02.856111 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.856057    2581 generic.go:356] "Generic (PLEG): container finished" podID="ee859d70-b16a-49d3-83eb-1b0d5f50e892" containerID="3f205ed779876216870243b79de138e626cfd9c960d1a789385dbc4d6c98b6c3" exitCode=0
Sep 06 19:30:02.856234 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.856107    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb" event={"ID":"ee859d70-b16a-49d3-83eb-1b0d5f50e892","Type":"ContainerDied","Data":"3f205ed779876216870243b79de138e626cfd9c960d1a789385dbc4d6c98b6c3"}
Sep 06 19:30:02.857520 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.857501    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"
Sep 06 19:30:02.857611 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.857531    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw" event={"ID":"afdfb772-3a6b-4d81-b691-0dcf6f13f004","Type":"ContainerDied","Data":"fc406e292919b682a7eafbf83dae321218e387a2c22276eb71deb0de7b385286"}
Sep 06 19:30:02.857611 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:02.857568    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc406e292919b682a7eafbf83dae321218e387a2c22276eb71deb0de7b385286"
Sep 06 19:30:03.013598 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:03.013563    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc"
Sep 06 19:30:03.095152 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:03.095110    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:03.864747 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:03.864716    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc"
Sep 06 19:30:03.865214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:03.864753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812050-7sbsc" event={"ID":"aa0a1aa2-13f9-4f7f-bff5-5020ddf5e816","Type":"ContainerDied","Data":"8870a68e68a576055c9726c2677254557b8d76c89283217c02c55d858152e0e3"}
Sep 06 19:30:03.865214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:03.864800    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8870a68e68a576055c9726c2677254557b8d76c89283217c02c55d858152e0e3"
Sep 06 19:30:04.022154 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:04.022132    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p"
Sep 06 19:30:04.046248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:04.046222    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb"
Sep 06 19:30:04.871414 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:04.871378    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p"
Sep 06 19:30:04.871414 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:04.871389    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p" event={"ID":"077e96db-14a7-4f7d-8517-34414ea741c9","Type":"ContainerDied","Data":"d0a6025e7b84a1cae50a06dafdeedcec2f748462726b4a535ef968eb4cb4c8b5"}
Sep 06 19:30:04.871853 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:04.871435    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0a6025e7b84a1cae50a06dafdeedcec2f748462726b4a535ef968eb4cb4c8b5"
Sep 06 19:30:04.872803 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:04.872786    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb"
Sep 06 19:30:04.872925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:04.872787    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb" event={"ID":"ee859d70-b16a-49d3-83eb-1b0d5f50e892","Type":"ContainerDied","Data":"c0d2303cb4080b9e75063b1aea931b821f32df64efa5078ef3a4a69d4bf1affc"}
Sep 06 19:30:04.872925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:04.872915    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0d2303cb4080b9e75063b1aea931b821f32df64efa5078ef3a4a69d4bf1affc"
Sep 06 19:30:05.463681 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:05.463642    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:05.996209 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:05.996170    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb"]
Sep 06 19:30:06.000812 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.000782    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-119d0999-29812050-wsqvb"]
Sep 06 19:30:06.001379 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.001356    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:06.067044 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.067010    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:06.167448 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.167412    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:06.178435 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.178388    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"]
Sep 06 19:30:06.182388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.182357    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-119d0999-8crsw"]
Sep 06 19:30:06.221876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.221838    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:06.275139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.271135    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afdfb772-3a6b-4d81-b691-0dcf6f13f004" path="/var/lib/kubelet/pods/afdfb772-3a6b-4d81-b691-0dcf6f13f004/volumes"
Sep 06 19:30:06.275139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.271597    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee859d70-b16a-49d3-83eb-1b0d5f50e892" path="/var/lib/kubelet/pods/ee859d70-b16a-49d3-83eb-1b0d5f50e892/volumes"
Sep 06 19:30:06.304764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.304728    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"]
Sep 06 19:30:06.304977 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.304953    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy" containerID="cri-o://f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec" gracePeriod=30
Sep 06 19:30:06.319136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.319052    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:06.950034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:06.949957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:07.050569 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:07.050531    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:07.148671 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:07.148632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:08.396574 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:08.396542    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:08.882424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:08.882387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:11.570023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.569995    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:30:11.667515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667477    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-token\") pod \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") "
Sep 06 19:30:11.667515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667518    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-envoy\") pod \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") "
Sep 06 19:30:11.667724 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667558    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-data\") pod \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") "
Sep 06 19:30:11.667724 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667600    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-socket\") pod \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") "
Sep 06 19:30:11.667724 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667687    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istiod-ca-cert\") pod \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") "
Sep 06 19:30:11.667878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667728    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-credential-socket\") pod \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") "
Sep 06 19:30:11.667878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667783    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-podinfo\") pod \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") "
Sep 06 19:30:11.667878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667813    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-certs\") pod \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") "
Sep 06 19:30:11.667878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc86x\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-kube-api-access-zc86x\") pod \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\" (UID: \"e91a1428-cf0a-4149-84d5-2ec93207c5bb\") "
Sep 06 19:30:11.668122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667908    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "e91a1428-cf0a-4149-84d5-2ec93207c5bb" (UID: "e91a1428-cf0a-4149-84d5-2ec93207c5bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:11.668122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.667938    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e91a1428-cf0a-4149-84d5-2ec93207c5bb" (UID: "e91a1428-cf0a-4149-84d5-2ec93207c5bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:11.668362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.668256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e91a1428-cf0a-4149-84d5-2ec93207c5bb" (UID: "e91a1428-cf0a-4149-84d5-2ec93207c5bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:11.668362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.668262    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e91a1428-cf0a-4149-84d5-2ec93207c5bb" (UID: "e91a1428-cf0a-4149-84d5-2ec93207c5bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:30:11.668362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.668353    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e91a1428-cf0a-4149-84d5-2ec93207c5bb" (UID: "e91a1428-cf0a-4149-84d5-2ec93207c5bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:11.668538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.668491    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:11.668538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.668515    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:11.669922 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.669896    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e91a1428-cf0a-4149-84d5-2ec93207c5bb" (UID: "e91a1428-cf0a-4149-84d5-2ec93207c5bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:11.670027 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.669995    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "e91a1428-cf0a-4149-84d5-2ec93207c5bb" (UID: "e91a1428-cf0a-4149-84d5-2ec93207c5bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:30:11.670613 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.670593    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-kube-api-access-zc86x" (OuterVolumeSpecName: "kube-api-access-zc86x") pod "e91a1428-cf0a-4149-84d5-2ec93207c5bb" (UID: "e91a1428-cf0a-4149-84d5-2ec93207c5bb"). InnerVolumeSpecName "kube-api-access-zc86x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:30:11.670688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.670608    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e91a1428-cf0a-4149-84d5-2ec93207c5bb" (UID: "e91a1428-cf0a-4149-84d5-2ec93207c5bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:30:11.769350 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.769308    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:11.769350 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.769343    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:11.769350 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.769355    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:11.769596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.769369    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:11.769596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.769381    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc86x\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-kube-api-access-zc86x\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:11.769596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.769392    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:11.769596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.769402    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e91a1428-cf0a-4149-84d5-2ec93207c5bb-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:11.926319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.926226    2581 generic.go:356] "Generic (PLEG): container finished" podID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerID="f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec" exitCode=0
Sep 06 19:30:11.926319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.926276    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"
Sep 06 19:30:11.926494 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.926318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" event={"ID":"e91a1428-cf0a-4149-84d5-2ec93207c5bb","Type":"ContainerDied","Data":"f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec"}
Sep 06 19:30:11.926494 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.926381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf" event={"ID":"e91a1428-cf0a-4149-84d5-2ec93207c5bb","Type":"ContainerDied","Data":"77108358edc60684d57f2a72419c1fdb705c06fd4bab3fe9468183a253e12b87"}
Sep 06 19:30:11.926494 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.926420    2581 scope.go:117] "RemoveContainer" containerID="f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec"
Sep 06 19:30:11.936376 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.936358    2581 scope.go:117] "RemoveContainer" containerID="b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d"
Sep 06 19:30:11.946283 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.946144    2581 scope.go:117] "RemoveContainer" containerID="f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec"
Sep 06 19:30:11.946759 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:30:11.946590    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec\": container with ID starting with f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec not found: ID does not exist" containerID="f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec"
Sep 06 19:30:11.946759 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.946623    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec"} err="failed to get container status \"f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec\": rpc error: code = NotFound desc = could not find container \"f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec\": container with ID starting with f4a1c90568a60f09e90762f62fa9b7da29dd3f14820d35555c3cd5c6908c42ec not found: ID does not exist"
Sep 06 19:30:11.946759 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.946648    2581 scope.go:117] "RemoveContainer" containerID="b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d"
Sep 06 19:30:11.946951 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:30:11.946927    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d\": container with ID starting with b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d not found: ID does not exist" containerID="b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d"
Sep 06 19:30:11.947006 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.946958    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d"} err="failed to get container status \"b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d\": rpc error: code = NotFound desc = could not find container \"b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d\": container with ID starting with b8ac50a63a2ecc5b58d9878f2cc72d4433303ee479e0fe62bdd4e94a3b38c61d not found: ID does not exist"
Sep 06 19:30:11.947986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.947970    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"]
Sep 06 19:30:11.950034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:11.950012    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-119d0999-openshift-default-6d8c7888b8-c77hf"]
Sep 06 19:30:12.267888 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:12.267798    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" path="/var/lib/kubelet/pods/e91a1428-cf0a-4149-84d5-2ec93207c5bb/volumes"
Sep 06 19:30:12.478855 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:12.478817    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"]
Sep 06 19:30:12.496519 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:12.496484    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:14.258222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:14.258188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:14.321235 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:14.321204    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:14.399675 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:14.399631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:14.501508 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:14.501479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:14.905380 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:14.905343    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:27.501219 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.501171    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2" podUID="6ca801dd-7adb-4406-b0e8-2b1343365c4a" containerName="main" containerID="cri-o://33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833" gracePeriod=45
Sep 06 19:30:27.756940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.756864    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:30:27.935698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.935659    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-model-cache\") pod \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") "
Sep 06 19:30:27.935879 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.935723    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tls-certs\") pod \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") "
Sep 06 19:30:27.935879 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.935833    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmh8f\" (UniqueName: \"kubernetes.io/projected/6ca801dd-7adb-4406-b0e8-2b1343365c4a-kube-api-access-zmh8f\") pod \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") "
Sep 06 19:30:27.935982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.935876    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tmp-dir\") pod \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") "
Sep 06 19:30:27.935982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.935927    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-dshm\") pod \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") "
Sep 06 19:30:27.935982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.935954    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-home\") pod \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\" (UID: \"6ca801dd-7adb-4406-b0e8-2b1343365c4a\") "
Sep 06 19:30:27.935982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.935966    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-model-cache" (OuterVolumeSpecName: "model-cache") pod "6ca801dd-7adb-4406-b0e8-2b1343365c4a" (UID: "6ca801dd-7adb-4406-b0e8-2b1343365c4a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:27.936260 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.936233    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6ca801dd-7adb-4406-b0e8-2b1343365c4a" (UID: "6ca801dd-7adb-4406-b0e8-2b1343365c4a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:27.936331 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.936283    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-model-cache\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:27.936378 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.936353    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-home" (OuterVolumeSpecName: "home") pod "6ca801dd-7adb-4406-b0e8-2b1343365c4a" (UID: "6ca801dd-7adb-4406-b0e8-2b1343365c4a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:27.938407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.938368    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ca801dd-7adb-4406-b0e8-2b1343365c4a-kube-api-access-zmh8f" (OuterVolumeSpecName: "kube-api-access-zmh8f") pod "6ca801dd-7adb-4406-b0e8-2b1343365c4a" (UID: "6ca801dd-7adb-4406-b0e8-2b1343365c4a"). InnerVolumeSpecName "kube-api-access-zmh8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:30:27.938541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.938514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6ca801dd-7adb-4406-b0e8-2b1343365c4a" (UID: "6ca801dd-7adb-4406-b0e8-2b1343365c4a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:30:27.938599 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:27.938529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-dshm" (OuterVolumeSpecName: "dshm") pod "6ca801dd-7adb-4406-b0e8-2b1343365c4a" (UID: "6ca801dd-7adb-4406-b0e8-2b1343365c4a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:28.018775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.018680    2581 generic.go:356] "Generic (PLEG): container finished" podID="6ca801dd-7adb-4406-b0e8-2b1343365c4a" containerID="33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833" exitCode=0
Sep 06 19:30:28.018775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.018748    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"
Sep 06 19:30:28.018775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.018758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2" event={"ID":"6ca801dd-7adb-4406-b0e8-2b1343365c4a","Type":"ContainerDied","Data":"33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833"}
Sep 06 19:30:28.019034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.018810    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2" event={"ID":"6ca801dd-7adb-4406-b0e8-2b1343365c4a","Type":"ContainerDied","Data":"0d2664a4ee612edaa2cece3ec59309d281448b47b918a1b6b4729667610a59c6"}
Sep 06 19:30:28.019034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.018828    2581 scope.go:117] "RemoveContainer" containerID="33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833"
Sep 06 19:30:28.032810 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.032789    2581 scope.go:117] "RemoveContainer" containerID="33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833"
Sep 06 19:30:28.033183 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:30:28.033161    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833\": container with ID starting with 33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833 not found: ID does not exist" containerID="33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833"
Sep 06 19:30:28.033268 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.033189    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833"} err="failed to get container status \"33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833\": rpc error: code = NotFound desc = could not find container \"33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833\": container with ID starting with 33d978059cba45ed57b4e41d44aa0c258eb2bc413231fdd2f2204610e9dc0833 not found: ID does not exist"
Sep 06 19:30:28.036880 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.036858    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tmp-dir\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:28.036880 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.036880    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-dshm\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:28.037079 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.036887    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca801dd-7adb-4406-b0e8-2b1343365c4a-home\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:28.037079 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.036897    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca801dd-7adb-4406-b0e8-2b1343365c4a-tls-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:28.037079 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.036905    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmh8f\" (UniqueName: \"kubernetes.io/projected/6ca801dd-7adb-4406-b0e8-2b1343365c4a-kube-api-access-zmh8f\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:28.045454 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.045424    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"]
Sep 06 19:30:28.047551 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.047524    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-4067c966-kserve-574c788468-vv6k2"]
Sep 06 19:30:28.153946 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.153910    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"]
Sep 06 19:30:28.154419 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154405    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa0a1aa2-13f9-4f7f-bff5-5020ddf5e816" containerName="cleanup"
Sep 06 19:30:28.154491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154421    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa0a1aa2-13f9-4f7f-bff5-5020ddf5e816" containerName="cleanup"
Sep 06 19:30:28.154491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154433    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="077e96db-14a7-4f7d-8517-34414ea741c9" containerName="cleanup"
Sep 06 19:30:28.154491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154438    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="077e96db-14a7-4f7d-8517-34414ea741c9" containerName="cleanup"
Sep 06 19:30:28.154491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154457    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee859d70-b16a-49d3-83eb-1b0d5f50e892" containerName="cleanup"
Sep 06 19:30:28.154491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154462    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee859d70-b16a-49d3-83eb-1b0d5f50e892" containerName="cleanup"
Sep 06 19:30:28.154491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154472    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy"
Sep 06 19:30:28.154491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154477    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy"
Sep 06 19:30:28.154491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154484    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154498    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154506    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ca801dd-7adb-4406-b0e8-2b1343365c4a" containerName="main"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154513    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ca801dd-7adb-4406-b0e8-2b1343365c4a" containerName="main"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154521    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afdfb772-3a6b-4d81-b691-0dcf6f13f004" containerName="revoke-keys"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154527    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="afdfb772-3a6b-4d81-b691-0dcf6f13f004" containerName="revoke-keys"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154592    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ca801dd-7adb-4406-b0e8-2b1343365c4a" containerName="main"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154599    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="afdfb772-3a6b-4d81-b691-0dcf6f13f004" containerName="revoke-keys"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154606    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="077e96db-14a7-4f7d-8517-34414ea741c9" containerName="cleanup"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154611    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee859d70-b16a-49d3-83eb-1b0d5f50e892" containerName="cleanup"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154617    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154622    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa0a1aa2-13f9-4f7f-bff5-5020ddf5e816" containerName="cleanup"
Sep 06 19:30:28.154757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.154627    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e91a1428-cf0a-4149-84d5-2ec93207c5bb" containerName="istio-proxy"
Sep 06 19:30:28.159330 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.159313    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.161430 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.161407    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5307a7\"/\"kube-root-ca.crt\""
Sep 06 19:30:28.161566 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.161407    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5307a7\"/\"openshift-service-ca.crt\""
Sep 06 19:30:28.161566 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.161461    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5307a7\"/\"default-dockercfg-nwbf2\""
Sep 06 19:30:28.161566 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.161464    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5307a7\"/\"test-model-5307a7-kserve-self-signed-certs\""
Sep 06 19:30:28.165285 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.165262    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"]
Sep 06 19:30:28.245698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.245669    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:28.266505 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.266474    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ca801dd-7adb-4406-b0e8-2b1343365c4a" path="/var/lib/kubelet/pods/6ca801dd-7adb-4406-b0e8-2b1343365c4a/volumes"
Sep 06 19:30:28.339218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.339125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-home\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.339218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.339178    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-tmp-dir\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.339410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.339269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-model-cache\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.339410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.339302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b14c50e-49fa-404a-939e-0f74f31ae229-tls-certs\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.339410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.339377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-dshm\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.339507 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.339416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh5px\" (UniqueName: \"kubernetes.io/projected/5b14c50e-49fa-404a-939e-0f74f31ae229-kube-api-access-zh5px\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.440610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.440572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-home\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.440797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.440624    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-tmp-dir\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.440797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.440677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-model-cache\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.440797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.440703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b14c50e-49fa-404a-939e-0f74f31ae229-tls-certs\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.440797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.440770    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-dshm\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.441007 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.440824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh5px\" (UniqueName: \"kubernetes.io/projected/5b14c50e-49fa-404a-939e-0f74f31ae229-kube-api-access-zh5px\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.441108 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.441050    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-home\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.441174 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.441115    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-model-cache\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.441174 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.441161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-tmp-dir\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.443214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.443193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-dshm\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.443410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.443396    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b14c50e-49fa-404a-939e-0f74f31ae229-tls-certs\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.448200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.448181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh5px\" (UniqueName: \"kubernetes.io/projected/5b14c50e-49fa-404a-939e-0f74f31ae229-kube-api-access-zh5px\") pod \"test-model-5307a7-kserve-59896ccc87-pdxzr\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") " pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.469864 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.469838    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:28.743448 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:28.743419    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"]
Sep 06 19:30:28.745407 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:30:28.745377    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b14c50e_49fa_404a_939e_0f74f31ae229.slice/crio-41c396455540a417546cec882b4540696f6c2126fc546c7df725461e48332973 WatchSource:0}: Error finding container 41c396455540a417546cec882b4540696f6c2126fc546c7df725461e48332973: Status 404 returned error can't find the container with id 41c396455540a417546cec882b4540696f6c2126fc546c7df725461e48332973
Sep 06 19:30:29.026564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:29.026462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr" event={"ID":"5b14c50e-49fa-404a-939e-0f74f31ae229","Type":"ContainerStarted","Data":"d0ce754c05a53ada2eb1f4fef2ccb4f803a5b9fc86bd12fd93726991b4a5aa19"}
Sep 06 19:30:29.026564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:29.026523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr" event={"ID":"5b14c50e-49fa-404a-939e-0f74f31ae229","Type":"ContainerStarted","Data":"41c396455540a417546cec882b4540696f6c2126fc546c7df725461e48332973"}
Sep 06 19:30:29.042169 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:29.042117    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr" podStartSLOduration=1.042098545 podStartE2EDuration="1.042098545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:30:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:30:29.040555505 +0000 UTC m=+1157.321008323" watchObservedRunningTime="2026-09-06 19:30:29.042098545 +0000 UTC m=+1157.322551338"
Sep 06 19:30:30.161833 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.161793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:30.642184 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.642154    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:30.742863 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.742826    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"]
Sep 06 19:30:30.746553 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.746535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"
Sep 06 19:30:30.748528 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.748509    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:30:30.748650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.748512    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:30:30.752984 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.752960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"]
Sep 06 19:30:30.761173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.761139    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/640ebce3-3761-429c-8448-d76e9930a28e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-21b01e-j9rkz\" (UID: \"640ebce3-3761-429c-8448-d76e9930a28e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"
Sep 06 19:30:30.862441 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.862385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/640ebce3-3761-429c-8448-d76e9930a28e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-21b01e-j9rkz\" (UID: \"640ebce3-3761-429c-8448-d76e9930a28e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"
Sep 06 19:30:30.863057 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.863034    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/640ebce3-3761-429c-8448-d76e9930a28e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-21b01e-j9rkz\" (UID: \"640ebce3-3761-429c-8448-d76e9930a28e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"
Sep 06 19:30:30.984787 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:30.984689    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:31.058715 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:31.058680    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"
Sep 06 19:30:31.331335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:31.331304    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"]
Sep 06 19:30:31.333115 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:30:31.333083    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod640ebce3_3761_429c_8448_d76e9930a28e.slice/crio-a118ec25e2fb1c8ec29dfd6fdc57c11f21b5d465354e071ae2efa8a43d48d60d WatchSource:0}: Error finding container a118ec25e2fb1c8ec29dfd6fdc57c11f21b5d465354e071ae2efa8a43d48d60d: Status 404 returned error can't find the container with id a118ec25e2fb1c8ec29dfd6fdc57c11f21b5d465354e071ae2efa8a43d48d60d
Sep 06 19:30:32.048188 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:32.048154    2581 generic.go:356] "Generic (PLEG): container finished" podID="640ebce3-3761-429c-8448-d76e9930a28e" containerID="c3aa14dbd4625aaa00638a8d59f865b4c6fc2af2db64dabaf13ed3d3822f2cbc" exitCode=0
Sep 06 19:30:32.048359 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:32.048239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz" event={"ID":"640ebce3-3761-429c-8448-d76e9930a28e","Type":"ContainerDied","Data":"c3aa14dbd4625aaa00638a8d59f865b4c6fc2af2db64dabaf13ed3d3822f2cbc"}
Sep 06 19:30:32.048359 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:32.048304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz" event={"ID":"640ebce3-3761-429c-8448-d76e9930a28e","Type":"ContainerStarted","Data":"a118ec25e2fb1c8ec29dfd6fdc57c11f21b5d465354e071ae2efa8a43d48d60d"}
Sep 06 19:30:33.197213 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:33.197189    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"
Sep 06 19:30:33.288509 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:33.288476    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/640ebce3-3761-429c-8448-d76e9930a28e-maas-api-service-ca\") pod \"640ebce3-3761-429c-8448-d76e9930a28e\" (UID: \"640ebce3-3761-429c-8448-d76e9930a28e\") "
Sep 06 19:30:33.288846 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:33.288824    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/640ebce3-3761-429c-8448-d76e9930a28e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "640ebce3-3761-429c-8448-d76e9930a28e" (UID: "640ebce3-3761-429c-8448-d76e9930a28e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:30:33.389582 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:33.389550    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/640ebce3-3761-429c-8448-d76e9930a28e-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:34.061979 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:34.061940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz" event={"ID":"640ebce3-3761-429c-8448-d76e9930a28e","Type":"ContainerDied","Data":"a118ec25e2fb1c8ec29dfd6fdc57c11f21b5d465354e071ae2efa8a43d48d60d"}
Sep 06 19:30:34.062155 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:34.061999    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a118ec25e2fb1c8ec29dfd6fdc57c11f21b5d465354e071ae2efa8a43d48d60d"
Sep 06 19:30:34.062155 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:34.062097    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"
Sep 06 19:30:38.470633 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:38.470580    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:38.470633 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:38.470649    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:38.483769 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:38.483742    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:39.106925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:39.106895    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:30:40.779456 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:40.779418    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:41.233879 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:41.233845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:41.246762 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:41.246730    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p"]
Sep 06 19:30:41.248816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:41.248794    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-21b01e-29812050-j269p"]
Sep 06 19:30:41.299382 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:41.299331    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:41.399465 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:41.399434    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:41.418402 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:41.418360    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"]
Sep 06 19:30:41.423491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:41.423465    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-21b01e-j9rkz"]
Sep 06 19:30:41.456538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:41.456502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:41.763824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:41.763785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:42.268134 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:42.268100    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="077e96db-14a7-4f7d-8517-34414ea741c9" path="/var/lib/kubelet/pods/077e96db-14a7-4f7d-8517-34414ea741c9/volumes"
Sep 06 19:30:42.268488 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:42.268442    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="640ebce3-3761-429c-8448-d76e9930a28e" path="/var/lib/kubelet/pods/640ebce3-3761-429c-8448-d76e9930a28e/volumes"
Sep 06 19:30:46.478482 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:46.478447    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:46.534525 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:46.534485    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:46.601722 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:46.601684    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:46.697194 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:46.697153    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:46.899182 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:46.899148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:48.301042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:48.301001    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:48.399665 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:48.399575    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:48.526325 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:48.526275    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"]
Sep 06 19:30:48.526757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:48.526545    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy" containerID="cri-o://1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b" gracePeriod=30
Sep 06 19:30:48.539584 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:48.539476    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:48.952946 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:48.952911    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:49.196496 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.196453    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"]
Sep 06 19:30:49.196928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.196912    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="640ebce3-3761-429c-8448-d76e9930a28e" containerName="revoke-keys"
Sep 06 19:30:49.196928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.196927    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="640ebce3-3761-429c-8448-d76e9930a28e" containerName="revoke-keys"
Sep 06 19:30:49.197106 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.197006    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="640ebce3-3761-429c-8448-d76e9930a28e" containerName="revoke-keys"
Sep 06 19:30:49.200588 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.200565    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.203846 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.203785    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-c6813a-openshift-default-dockercfg-z8k64\""
Sep 06 19:30:49.212995 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.212972    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"]
Sep 06 19:30:49.230257 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.230218    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:49.352697 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.352661    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-data\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.352697 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.352704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-envoy\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.353139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.352732    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-credential-socket\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.353139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.352802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-socket\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.353139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.352834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istiod-ca-cert\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.353139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.352871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-podinfo\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.353139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.352982    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-certs\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.353139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.353018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxg8r\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-kube-api-access-kxg8r\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.353139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.353053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-token\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.382436 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.382407    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:49.454045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.453942    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-certs\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxg8r\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-kube-api-access-kxg8r\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-token\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-data\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-envoy\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-credential-socket\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-socket\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istiod-ca-cert\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-podinfo\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454708 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454430    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-certs\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454708 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454510    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-data\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454708 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-credential-socket\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454854 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-socket\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.454969 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.454948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istiod-ca-cert\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.456641 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.456620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-envoy\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.457006 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.456989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-podinfo\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.465203 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.465181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-token\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.465411 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.465391    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxg8r\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-kube-api-access-kxg8r\") pod \"e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") " pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.511906 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.511874    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:49.787576 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.787546    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"]
Sep 06 19:30:49.789234 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:30:49.789210    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc48602bf_ce7e_4e74_9a3a_fe50b1c45d89.slice/crio-75e23ace063bf8534d292928b7d6e81c67369705f033ed0c514f9c9e531bef51 WatchSource:0}: Error finding container 75e23ace063bf8534d292928b7d6e81c67369705f033ed0c514f9c9e531bef51: Status 404 returned error can't find the container with id 75e23ace063bf8534d292928b7d6e81c67369705f033ed0c514f9c9e531bef51
Sep 06 19:30:49.791614 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.791582    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:30:49.791688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.791649    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:30:49.791688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:49.791677    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:30:50.165098 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:50.165042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" event={"ID":"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89","Type":"ContainerStarted","Data":"c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1"}
Sep 06 19:30:50.165278 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:50.165159    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" event={"ID":"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89","Type":"ContainerStarted","Data":"75e23ace063bf8534d292928b7d6e81c67369705f033ed0c514f9c9e531bef51"}
Sep 06 19:30:50.183387 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:50.183332    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" podStartSLOduration=1.183316069 podStartE2EDuration="1.183316069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:30:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:30:50.182365945 +0000 UTC m=+1178.462818741" watchObservedRunningTime="2026-09-06 19:30:50.183316069 +0000 UTC m=+1178.463768863"
Sep 06 19:30:50.512422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:50.512332    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:50.517551 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:50.517521    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:51.170749 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:51.170715    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:51.171748 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:51.171728    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:30:52.012212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:52.012175    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:52.112272 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:52.112232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:53.290319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.290287    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:53.384859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.384825    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:53.484782 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.484753    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:53.747459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.747412    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:53.786664 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.786634    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:30:53.811355 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.811273    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:30:53.904866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.904826    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-credential-socket\") pod \"6c070e3c-e087-4a77-88e5-db650bce4f57\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") "
Sep 06 19:30:53.905113 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.904896    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-podinfo\") pod \"6c070e3c-e087-4a77-88e5-db650bce4f57\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") "
Sep 06 19:30:53.905113 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.904924    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-token\") pod \"6c070e3c-e087-4a77-88e5-db650bce4f57\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") "
Sep 06 19:30:53.905113 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.905023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-certs\") pod \"6c070e3c-e087-4a77-88e5-db650bce4f57\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") "
Sep 06 19:30:53.905364 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.905171    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6c070e3c-e087-4a77-88e5-db650bce4f57" (UID: "6c070e3c-e087-4a77-88e5-db650bce4f57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:53.905867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.905825    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6c070e3c-e087-4a77-88e5-db650bce4f57" (UID: "6c070e3c-e087-4a77-88e5-db650bce4f57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:53.906089 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.906032    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6c070e3c-e087-4a77-88e5-db650bce4f57" (UID: "6c070e3c-e087-4a77-88e5-db650bce4f57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:53.906221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.906203    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-socket\") pod \"6c070e3c-e087-4a77-88e5-db650bce4f57\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") "
Sep 06 19:30:53.906349 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.906334    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c070e3c-e087-4a77-88e5-db650bce4f57-istiod-ca-cert\") pod \"6c070e3c-e087-4a77-88e5-db650bce4f57\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") "
Sep 06 19:30:53.906480 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.906464    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-envoy\") pod \"6c070e3c-e087-4a77-88e5-db650bce4f57\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") "
Sep 06 19:30:53.906626 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.906611    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-data\") pod \"6c070e3c-e087-4a77-88e5-db650bce4f57\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") "
Sep 06 19:30:53.906760 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.906746    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkh2x\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-kube-api-access-xkh2x\") pod \"6c070e3c-e087-4a77-88e5-db650bce4f57\" (UID: \"6c070e3c-e087-4a77-88e5-db650bce4f57\") "
Sep 06 19:30:53.907105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.906665    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c070e3c-e087-4a77-88e5-db650bce4f57-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6c070e3c-e087-4a77-88e5-db650bce4f57" (UID: "6c070e3c-e087-4a77-88e5-db650bce4f57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:30:53.907105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.907049    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-data" (OuterVolumeSpecName: "istio-data") pod "6c070e3c-e087-4a77-88e5-db650bce4f57" (UID: "6c070e3c-e087-4a77-88e5-db650bce4f57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:53.907794 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.907775    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:53.907908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.907895    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:53.907999 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.907986    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:53.908173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.908158    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:53.908280 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.908267    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c070e3c-e087-4a77-88e5-db650bce4f57-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:53.912267 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.912221    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6c070e3c-e087-4a77-88e5-db650bce4f57" (UID: "6c070e3c-e087-4a77-88e5-db650bce4f57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:30:53.912540 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.912406    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6c070e3c-e087-4a77-88e5-db650bce4f57" (UID: "6c070e3c-e087-4a77-88e5-db650bce4f57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:30:53.912540 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.912487    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-kube-api-access-xkh2x" (OuterVolumeSpecName: "kube-api-access-xkh2x") pod "6c070e3c-e087-4a77-88e5-db650bce4f57" (UID: "6c070e3c-e087-4a77-88e5-db650bce4f57"). InnerVolumeSpecName "kube-api-access-xkh2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:30:53.915460 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:53.915432    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-token" (OuterVolumeSpecName: "istio-token") pod "6c070e3c-e087-4a77-88e5-db650bce4f57" (UID: "6c070e3c-e087-4a77-88e5-db650bce4f57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:30:54.009140 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.009096    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:54.009140 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.009127    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:54.009140 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.009141    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c070e3c-e087-4a77-88e5-db650bce4f57-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:54.009459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.009152    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkh2x\" (UniqueName: \"kubernetes.io/projected/6c070e3c-e087-4a77-88e5-db650bce4f57-kube-api-access-xkh2x\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:30:54.189770 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.189734    2581 generic.go:356] "Generic (PLEG): container finished" podID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerID="1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b" exitCode=0
Sep 06 19:30:54.189951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.189799    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"
Sep 06 19:30:54.189951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.189818    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" event={"ID":"6c070e3c-e087-4a77-88e5-db650bce4f57","Type":"ContainerDied","Data":"1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b"}
Sep 06 19:30:54.189951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.189887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg" event={"ID":"6c070e3c-e087-4a77-88e5-db650bce4f57","Type":"ContainerDied","Data":"af58585856080bb117738679c935e9a0e5ea05f115740b55ec803417b2036ad4"}
Sep 06 19:30:54.189951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.189925    2581 scope.go:117] "RemoveContainer" containerID="1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b"
Sep 06 19:30:54.203336 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.203314    2581 scope.go:117] "RemoveContainer" containerID="3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d"
Sep 06 19:30:54.215429 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.215342    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"]
Sep 06 19:30:54.215540 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.215523    2581 scope.go:117] "RemoveContainer" containerID="1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b"
Sep 06 19:30:54.216054 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:30:54.215910    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b\": container with ID starting with 1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b not found: ID does not exist" containerID="1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b"
Sep 06 19:30:54.216054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.215949    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b"} err="failed to get container status \"1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b\": rpc error: code = NotFound desc = could not find container \"1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b\": container with ID starting with 1d0cffb1f80de2c0ba2cb5c7bead6208480ab48833749bb87ac23ae3bbd6c06b not found: ID does not exist"
Sep 06 19:30:54.216054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.215976    2581 scope.go:117] "RemoveContainer" containerID="3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d"
Sep 06 19:30:54.216998 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:30:54.216589    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d\": container with ID starting with 3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d not found: ID does not exist" containerID="3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d"
Sep 06 19:30:54.216998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.216620    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d"} err="failed to get container status \"3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d\": rpc error: code = NotFound desc = could not find container \"3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d\": container with ID starting with 3b965f977d055fc82a508722d6c6d4b8b2f9b109b85b4179e8539d569be3ef1d not found: ID does not exist"
Sep 06 19:30:54.219077 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.219040    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-21b01e-openshift-default-5874ff95fb-6chgg"]
Sep 06 19:30:54.266023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:30:54.265983    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" path="/var/lib/kubelet/pods/6c070e3c-e087-4a77-88e5-db650bce4f57/volumes"
Sep 06 19:31:01.542565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:01.542528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:02.444048 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:02.444008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:08.232324 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:08.232285    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:09.593523 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:09.593491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:10.294838 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:10.294805    2581 generic.go:356] "Generic (PLEG): container finished" podID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerID="c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1" exitCode=0
Sep 06 19:31:10.295016 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:10.294886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" event={"ID":"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89","Type":"ContainerDied","Data":"c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1"}
Sep 06 19:31:10.295391 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:10.295376    2581 scope.go:117] "RemoveContainer" containerID="c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1"
Sep 06 19:31:10.296552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:10.296525    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:31:10.296615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:10.296592    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:31:10.296651 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:10.296624    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:31:11.307558 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:11.307514    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" event={"ID":"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89","Type":"ContainerStarted","Data":"86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5"}
Sep 06 19:31:11.512692 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:11.512655    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:31:11.513134 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:11.513015    2581 patch_prober.go:28] interesting pod/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 06 19:31:11.513347 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:11.513322    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 06 19:31:12.281794 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:12.281759    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:31:12.284787 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:12.284757    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:31:12.313525 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:12.313491    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:31:12.318102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:12.318052    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:31:12.513393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:12.513351    2581 patch_prober.go:28] interesting pod/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 06 19:31:12.513602 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:12.513422    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 06 19:31:13.513444 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:13.513401    2581 patch_prober.go:28] interesting pod/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 06 19:31:13.513859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:13.513492    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 06 19:31:14.512918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:14.512874    2581 patch_prober.go:28] interesting pod/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 06 19:31:14.513154 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:14.512967    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 06 19:31:15.212227 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:15.212163    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:15.512970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:15.512860    2581 patch_prober.go:28] interesting pod/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 06 19:31:15.512970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:15.512953    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 06 19:31:16.513407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:16.513351    2581 patch_prober.go:28] interesting pod/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 06 19:31:16.513801 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:16.513476    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 06 19:31:16.941594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:16.941557    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:17.516447 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:17.516415    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:31:17.516861 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:17.516709    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:31:17.517625 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:17.517607    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:31:18.890636 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:18.890596    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:19.639379 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:19.639340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:25.214538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:25.214462    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:25.316291 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:25.316256    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:25.417631 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:25.417593    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:25.513419 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:25.513336    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:26.107734 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:26.107677    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:26.203137 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:26.203092    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:26.302861 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:26.302828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:27.705765 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:27.705732    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:29.365646 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:29.365608    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:33.413925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:33.413885    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:33.471959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:33.471925    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:34.070731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:34.070701    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:39.839191 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:39.839146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:39.895874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:39.895836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:40.061140 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.061105    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:40.316951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.316913    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"]
Sep 06 19:31:40.317561 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.317537    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy"
Sep 06 19:31:40.317688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.317624    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy"
Sep 06 19:31:40.317688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.317677    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy"
Sep 06 19:31:40.317688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.317686    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy"
Sep 06 19:31:40.317854 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.317805    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy"
Sep 06 19:31:40.321122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.321101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.323208 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.323187    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-032bc360-openshift-default-dockercfg-z7fdt\""
Sep 06 19:31:40.332557 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.332530    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"]
Sep 06 19:31:40.347520 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.347485    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:40.396411 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.396373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-data\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.396411 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.396414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-podinfo\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.396644 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.396468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39f42f82-7e87-48f1-98aa-0b212de177e1-istiod-ca-cert\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.396644 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.396579    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-credential-socket\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.396726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.396671    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-certs\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.396726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.396704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-socket\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.396807 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.396787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-token\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.396844 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.396814    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbpq9\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-kube-api-access-cbpq9\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.396894 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.396842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-envoy\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.497987 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.497944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-certs\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.497997    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-socket\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-token\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbpq9\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-kube-api-access-cbpq9\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-envoy\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-data\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-podinfo\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39f42f82-7e87-48f1-98aa-0b212de177e1-istiod-ca-cert\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-credential-socket\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-socket\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498704 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-certs\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498704 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-data\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.498800 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.498783    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-credential-socket\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.499242 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.499223    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39f42f82-7e87-48f1-98aa-0b212de177e1-istiod-ca-cert\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.500757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.500742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-envoy\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.500997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.500979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-podinfo\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.506177 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.506151    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-token\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.506361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.506343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbpq9\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-kube-api-access-cbpq9\") pod \"e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") " pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.632812 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.632778    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:40.885742 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.885658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:40.912496 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.912471    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"]
Sep 06 19:31:40.914224 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:31:40.914198    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39f42f82_7e87_48f1_98aa_0b212de177e1.slice/crio-b347f9c740daa383b83aeba650107c2c1a8e7be53f6d0d7e9e727faedcaadc15 WatchSource:0}: Error finding container b347f9c740daa383b83aeba650107c2c1a8e7be53f6d0d7e9e727faedcaadc15: Status 404 returned error can't find the container with id b347f9c740daa383b83aeba650107c2c1a8e7be53f6d0d7e9e727faedcaadc15
Sep 06 19:31:40.916978 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.916944    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:31:40.917135 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.917028    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:31:40.917135 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:40.917084    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:31:41.499734 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:41.499688    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" event={"ID":"39f42f82-7e87-48f1-98aa-0b212de177e1","Type":"ContainerStarted","Data":"38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671"}
Sep 06 19:31:41.499946 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:41.499751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" event={"ID":"39f42f82-7e87-48f1-98aa-0b212de177e1","Type":"ContainerStarted","Data":"b347f9c740daa383b83aeba650107c2c1a8e7be53f6d0d7e9e727faedcaadc15"}
Sep 06 19:31:41.518978 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:41.518928    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" podStartSLOduration=1.518910164 podStartE2EDuration="1.518910164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:31:41.516862963 +0000 UTC m=+1229.797315783" watchObservedRunningTime="2026-09-06 19:31:41.518910164 +0000 UTC m=+1229.799362957"
Sep 06 19:31:41.633678 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:41.633643    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:41.642300 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:41.642271    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:42.506153 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:42.506120    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:42.507319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:42.507297    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:31:43.379960 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:43.379929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:43.513567 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:43.513541    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:43.899089 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:43.899047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:44.507003 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:44.506963    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:44.573197 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:44.573163    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:44.673472 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:44.673439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:44.972505 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:44.972476    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:45.072490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:45.072457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:31:49.323829 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:31:49.323800    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:00.946390 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:00.946358    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:01.061178 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.061139    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"]
Sep 06 19:32:01.061711 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.061687    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c070e3c-e087-4a77-88e5-db650bce4f57" containerName="istio-proxy"
Sep 06 19:32:01.065719 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.065700    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"
Sep 06 19:32:01.068914 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.068883    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:32:01.069176 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.069142    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:32:01.071511 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.071475    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"]
Sep 06 19:32:01.233459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.233357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a470e0-9230-4cb4-ac7b-173c8bb29649-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv\" (UID: \"f2a470e0-9230-4cb4-ac7b-173c8bb29649\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"
Sep 06 19:32:01.282041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.282001    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:01.334473 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.334423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a470e0-9230-4cb4-ac7b-173c8bb29649-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv\" (UID: \"f2a470e0-9230-4cb4-ac7b-173c8bb29649\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"
Sep 06 19:32:01.335212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.335188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a470e0-9230-4cb4-ac7b-173c8bb29649-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv\" (UID: \"f2a470e0-9230-4cb4-ac7b-173c8bb29649\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"
Sep 06 19:32:01.381682 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.381645    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"
Sep 06 19:32:01.666187 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:01.666162    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"]
Sep 06 19:32:01.667924 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:32:01.667893    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a470e0_9230_4cb4_ac7b_173c8bb29649.slice/crio-53969924c5952af7b4e305f464ec7db762fa6161974cb8f0064b57fcb5cba742 WatchSource:0}: Error finding container 53969924c5952af7b4e305f464ec7db762fa6161974cb8f0064b57fcb5cba742: Status 404 returned error can't find the container with id 53969924c5952af7b4e305f464ec7db762fa6161974cb8f0064b57fcb5cba742
Sep 06 19:32:02.631349 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:02.631304    2581 generic.go:356] "Generic (PLEG): container finished" podID="f2a470e0-9230-4cb4-ac7b-173c8bb29649" containerID="eabf539bb7259f9cd6ceea9ddd356a27035e350b63635ea32ebfec6847cd336f" exitCode=0
Sep 06 19:32:02.631723 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:02.631361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv" event={"ID":"f2a470e0-9230-4cb4-ac7b-173c8bb29649","Type":"ContainerDied","Data":"eabf539bb7259f9cd6ceea9ddd356a27035e350b63635ea32ebfec6847cd336f"}
Sep 06 19:32:02.631723 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:02.631447    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv" event={"ID":"f2a470e0-9230-4cb4-ac7b-173c8bb29649","Type":"ContainerStarted","Data":"53969924c5952af7b4e305f464ec7db762fa6161974cb8f0064b57fcb5cba742"}
Sep 06 19:32:03.638923 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.638892    2581 generic.go:356] "Generic (PLEG): container finished" podID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerID="38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671" exitCode=0
Sep 06 19:32:03.639418 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.638973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" event={"ID":"39f42f82-7e87-48f1-98aa-0b212de177e1","Type":"ContainerDied","Data":"38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671"}
Sep 06 19:32:03.639775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.639589    2581 scope.go:117] "RemoveContainer" containerID="38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671"
Sep 06 19:32:03.640769 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.640741    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:32:03.640880 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.640825    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:32:03.640880 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.640874    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:32:03.800602 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.800575    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"
Sep 06 19:32:03.960668 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.960572    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a470e0-9230-4cb4-ac7b-173c8bb29649-maas-api-service-ca\") pod \"f2a470e0-9230-4cb4-ac7b-173c8bb29649\" (UID: \"f2a470e0-9230-4cb4-ac7b-173c8bb29649\") "
Sep 06 19:32:03.960942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.960918    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2a470e0-9230-4cb4-ac7b-173c8bb29649-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f2a470e0-9230-4cb4-ac7b-173c8bb29649" (UID: "f2a470e0-9230-4cb4-ac7b-173c8bb29649"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:32:03.961013 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:03.960998    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a470e0-9230-4cb4-ac7b-173c8bb29649-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:04.647327 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:04.647284    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" event={"ID":"39f42f82-7e87-48f1-98aa-0b212de177e1","Type":"ContainerStarted","Data":"93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785"}
Sep 06 19:32:04.648796 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:04.648775    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv" event={"ID":"f2a470e0-9230-4cb4-ac7b-173c8bb29649","Type":"ContainerDied","Data":"53969924c5952af7b4e305f464ec7db762fa6161974cb8f0064b57fcb5cba742"}
Sep 06 19:32:04.648796 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:04.648787    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"
Sep 06 19:32:04.648956 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:04.648811    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53969924c5952af7b4e305f464ec7db762fa6161974cb8f0064b57fcb5cba742"
Sep 06 19:32:05.345336 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:05.345306    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:05.633331 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:05.633292    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:32:05.633620 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:05.633592    2581 patch_prober.go:28] interesting pod/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 19:32:05.633715 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:05.633668    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 19:32:06.633256 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:06.633215    2581 patch_prober.go:28] interesting pod/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 19:32:06.633764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:06.633289    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 19:32:07.633506 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:07.633469    2581 patch_prober.go:28] interesting pod/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 19:32:07.633927 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:07.633544    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 19:32:08.268102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:08.268049    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:08.633537 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:08.633505    2581 patch_prober.go:28] interesting pod/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 19:32:08.633957 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:08.633574    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 19:32:08.872634 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:08.872581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:08.973490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:08.973404    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:09.283827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:09.283730    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:09.633271 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:09.633237    2581 patch_prober.go:28] interesting pod/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 19:32:09.633442 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:09.633293    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 19:32:10.638243 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:10.638205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:32:10.638705 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:10.638593    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:32:10.639366 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:10.639351    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:32:11.173708 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:11.173678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:11.683196 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:11.683162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:11.740449 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:11.740412    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:11.796838 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:11.796809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:11.860999 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:11.860964    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"]
Sep 06 19:32:11.862632 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:11.862605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:11.863377 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:11.863359    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c6813a-6ddnv"]
Sep 06 19:32:11.919094 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:11.919028    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:12.194538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:12.194508    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:12.266743 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:12.266713    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2a470e0-9230-4cb4-ac7b-173c8bb29649" path="/var/lib/kubelet/pods/f2a470e0-9230-4cb4-ac7b-173c8bb29649/volumes"
Sep 06 19:32:14.941383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:14.941346    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:14.999320 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:14.999263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:15.269438 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.269352    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"]
Sep 06 19:32:15.269832 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.269814    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2a470e0-9230-4cb4-ac7b-173c8bb29649" containerName="revoke-keys"
Sep 06 19:32:15.269897 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.269837    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2a470e0-9230-4cb4-ac7b-173c8bb29649" containerName="revoke-keys"
Sep 06 19:32:15.269984 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.269972    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2a470e0-9230-4cb4-ac7b-173c8bb29649" containerName="revoke-keys"
Sep 06 19:32:15.274126 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.274108    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"
Sep 06 19:32:15.276942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.276763    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:32:15.276942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.276806    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:32:15.279378 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.279354    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"]
Sep 06 19:32:15.388616 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.388586    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90c9f67c-4d70-4e25-b499-b28fb55baf77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-032bc360-gcgz5\" (UID: \"90c9f67c-4d70-4e25-b499-b28fb55baf77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"
Sep 06 19:32:15.475504 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.475471    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:15.489795 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.489764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90c9f67c-4d70-4e25-b499-b28fb55baf77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-032bc360-gcgz5\" (UID: \"90c9f67c-4d70-4e25-b499-b28fb55baf77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"
Sep 06 19:32:15.490458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.490436    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90c9f67c-4d70-4e25-b499-b28fb55baf77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-032bc360-gcgz5\" (UID: \"90c9f67c-4d70-4e25-b499-b28fb55baf77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"
Sep 06 19:32:15.586577 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.586479    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"
Sep 06 19:32:15.871674 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:15.871648    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"]
Sep 06 19:32:15.874305 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:32:15.874266    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c9f67c_4d70_4e25_b499_b28fb55baf77.slice/crio-8677fc6312371daeefd69a52875e03250de745ec36ac62a1c9d4466108070fd1 WatchSource:0}: Error finding container 8677fc6312371daeefd69a52875e03250de745ec36ac62a1c9d4466108070fd1: Status 404 returned error can't find the container with id 8677fc6312371daeefd69a52875e03250de745ec36ac62a1c9d4466108070fd1
Sep 06 19:32:16.730976 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:16.730942    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5" event={"ID":"90c9f67c-4d70-4e25-b499-b28fb55baf77","Type":"ContainerStarted","Data":"beda8d46f44a79b50049aef8a3667de79f795433bdae3bdc3b6e13bbee704c75"}
Sep 06 19:32:16.731390 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:16.730996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5" event={"ID":"90c9f67c-4d70-4e25-b499-b28fb55baf77","Type":"ContainerStarted","Data":"8677fc6312371daeefd69a52875e03250de745ec36ac62a1c9d4466108070fd1"}
Sep 06 19:32:16.746442 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:16.746363    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5" podStartSLOduration=1.746339561 podStartE2EDuration="1.746339561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:32:16.744411565 +0000 UTC m=+1265.024864359" watchObservedRunningTime="2026-09-06 19:32:16.746339561 +0000 UTC m=+1265.026792356"
Sep 06 19:32:17.274394 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:17.274359    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:17.738532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:17.738488    2581 generic.go:356] "Generic (PLEG): container finished" podID="90c9f67c-4d70-4e25-b499-b28fb55baf77" containerID="beda8d46f44a79b50049aef8a3667de79f795433bdae3bdc3b6e13bbee704c75" exitCode=0
Sep 06 19:32:17.738940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:17.738570    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5" event={"ID":"90c9f67c-4d70-4e25-b499-b28fb55baf77","Type":"ContainerDied","Data":"beda8d46f44a79b50049aef8a3667de79f795433bdae3bdc3b6e13bbee704c75"}
Sep 06 19:32:18.888712 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:18.888688    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"
Sep 06 19:32:19.036018 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:19.035917    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90c9f67c-4d70-4e25-b499-b28fb55baf77-maas-api-service-ca\") pod \"90c9f67c-4d70-4e25-b499-b28fb55baf77\" (UID: \"90c9f67c-4d70-4e25-b499-b28fb55baf77\") "
Sep 06 19:32:19.036383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:19.036361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90c9f67c-4d70-4e25-b499-b28fb55baf77-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "90c9f67c-4d70-4e25-b499-b28fb55baf77" (UID: "90c9f67c-4d70-4e25-b499-b28fb55baf77"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:32:19.137597 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:19.137562    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90c9f67c-4d70-4e25-b499-b28fb55baf77-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:19.751420 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:19.751383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5" event={"ID":"90c9f67c-4d70-4e25-b499-b28fb55baf77","Type":"ContainerDied","Data":"8677fc6312371daeefd69a52875e03250de745ec36ac62a1c9d4466108070fd1"}
Sep 06 19:32:19.751420 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:19.751417    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"
Sep 06 19:32:19.751637 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:19.751434    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8677fc6312371daeefd69a52875e03250de745ec36ac62a1c9d4466108070fd1"
Sep 06 19:32:22.764552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:22.764514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:22.986663 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:22.986629    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"]
Sep 06 19:32:22.986878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:22.986851    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy" containerID="cri-o://86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5" gracePeriod=30
Sep 06 19:32:23.007896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.007857    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:23.431825 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.431790    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:23.696428 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.696339    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"]
Sep 06 19:32:23.701677 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.701635    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90c9f67c-4d70-4e25-b499-b28fb55baf77" containerName="revoke-keys"
Sep 06 19:32:23.701677 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.701675    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="90c9f67c-4d70-4e25-b499-b28fb55baf77" containerName="revoke-keys"
Sep 06 19:32:23.701960 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.701944    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="90c9f67c-4d70-4e25-b499-b28fb55baf77" containerName="revoke-keys"
Sep 06 19:32:23.708083 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.708032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.710333 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.710305    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-477eea-openshift-default-dockercfg-pc46v\""
Sep 06 19:32:23.711290 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.711261    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"]
Sep 06 19:32:23.721232 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.721204    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:23.788577 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.788533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-credential-socket\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.789041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.788582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-envoy\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.789041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.788621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-podinfo\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.789041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.788657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-certs\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.789041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.788741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istiod-ca-cert\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.789041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.788817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-token\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.789041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.788894    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksxrg\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-kube-api-access-ksxrg\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.789041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.788922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-data\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.789041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.788959    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-socket\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-credential-socket\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890334 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-envoy\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890334 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-podinfo\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890334 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-certs\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890334 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istiod-ca-cert\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890334 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-token\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890334 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksxrg\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-kube-api-access-ksxrg\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890685 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890353    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-data\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890685 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-socket\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890685 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-credential-socket\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890693    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-certs\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890816 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890733    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-socket\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.890989 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.890968    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-data\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.891270 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.891244    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istiod-ca-cert\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.893105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.893057    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-envoy\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.893503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.893482    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-podinfo\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.898579 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.898549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-token\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.898684 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.898624    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksxrg\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-kube-api-access-ksxrg\") pod \"e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") " pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:23.958982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:23.958845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:24.022794 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:24.022764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:24.301456 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:24.301423    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"]
Sep 06 19:32:24.302803 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:32:24.302775    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bfd79b5_3c1a_4f2f_b2bf_ab671194351b.slice/crio-587835b06de64d4e1eb2c24af2d7b4398ffebce82fd656ef17a30d028bcdf75e WatchSource:0}: Error finding container 587835b06de64d4e1eb2c24af2d7b4398ffebce82fd656ef17a30d028bcdf75e: Status 404 returned error can't find the container with id 587835b06de64d4e1eb2c24af2d7b4398ffebce82fd656ef17a30d028bcdf75e
Sep 06 19:32:24.305396 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:24.305367    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:32:24.305483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:24.305432    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:32:24.305483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:24.305462    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:32:24.784077 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:24.784029    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv" event={"ID":"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b","Type":"ContainerStarted","Data":"cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d"}
Sep 06 19:32:24.784382 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:24.784101    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv" event={"ID":"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b","Type":"ContainerStarted","Data":"587835b06de64d4e1eb2c24af2d7b4398ffebce82fd656ef17a30d028bcdf75e"}
Sep 06 19:32:24.802412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:24.802344    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv" podStartSLOduration=1.802324566 podStartE2EDuration="1.802324566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:32:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:32:24.801226753 +0000 UTC m=+1273.081679595" watchObservedRunningTime="2026-09-06 19:32:24.802324566 +0000 UTC m=+1273.082777360"
Sep 06 19:32:25.023871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:25.023835    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:25.029280 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:25.029250    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:25.067212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:25.067112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:25.168663 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:25.168627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:25.267756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:25.267726    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:25.792476 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:25.792439    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:25.793548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:25.793524    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:25.908585 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:25.908545    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:26.009265 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:26.009228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:26.080077 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:26.079592    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"]
Sep 06 19:32:26.081945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:26.081914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:26.082599 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:26.082579    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-032bc360-gcgz5"]
Sep 06 19:32:26.139148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:26.139117    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:26.276047 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:26.275817    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90c9f67c-4d70-4e25-b499-b28fb55baf77" path="/var/lib/kubelet/pods/90c9f67c-4d70-4e25-b499-b28fb55baf77/volumes"
Sep 06 19:32:26.425705 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:26.425674    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:28.240851 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.240824    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:32:28.341940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.341806    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-certs\") pod \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") "
Sep 06 19:32:28.341940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.341931    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-envoy\") pod \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") "
Sep 06 19:32:28.342275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.341985    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxg8r\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-kube-api-access-kxg8r\") pod \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") "
Sep 06 19:32:28.342275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342036    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-socket\") pod \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") "
Sep 06 19:32:28.342379 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342292    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-token\") pod \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") "
Sep 06 19:32:28.342379 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342327    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-data\") pod \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") "
Sep 06 19:32:28.342481 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342446    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istiod-ca-cert\") pod \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") "
Sep 06 19:32:28.342583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342499    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-podinfo\") pod \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") "
Sep 06 19:32:28.342583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342538    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-credential-socket\") pod \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\" (UID: \"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89\") "
Sep 06 19:32:28.342583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342543    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" (UID: "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:28.342999 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342973    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" (UID: "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:32:28.342999 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342986    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" (UID: "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:28.343213 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.342977    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:28.343276 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.343252    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-data" (OuterVolumeSpecName: "istio-data") pod "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" (UID: "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:28.343368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.343343    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" (UID: "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:28.345301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.345270    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" (UID: "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:32:28.345301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.345277    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-token" (OuterVolumeSpecName: "istio-token") pod "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" (UID: "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:32:28.345811 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.345781    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-kube-api-access-kxg8r" (OuterVolumeSpecName: "kube-api-access-kxg8r") pod "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" (UID: "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89"). InnerVolumeSpecName "kube-api-access-kxg8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:32:28.345889 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.345855    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" (UID: "c48602bf-ce7e-4e74-9a3a-fe50b1c45d89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:28.444610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.444572    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:28.444610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.444606    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:28.444610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.444617    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:28.444872 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.444626    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:28.444872 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.444635    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxg8r\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-kube-api-access-kxg8r\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:28.444872 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.444644    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:28.444872 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.444651    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:28.444872 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.444659    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:28.811207 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.811172    2581 generic.go:356] "Generic (PLEG): container finished" podID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerID="86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5" exitCode=0
Sep 06 19:32:28.811410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.811225    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"
Sep 06 19:32:28.811410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.811261    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" event={"ID":"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89","Type":"ContainerDied","Data":"86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5"}
Sep 06 19:32:28.811410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.811325    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth" event={"ID":"c48602bf-ce7e-4e74-9a3a-fe50b1c45d89","Type":"ContainerDied","Data":"75e23ace063bf8534d292928b7d6e81c67369705f033ed0c514f9c9e531bef51"}
Sep 06 19:32:28.811410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.811355    2581 scope.go:117] "RemoveContainer" containerID="86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5"
Sep 06 19:32:28.821991 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.821974    2581 scope.go:117] "RemoveContainer" containerID="c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1"
Sep 06 19:32:28.831561 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.831508    2581 scope.go:117] "RemoveContainer" containerID="86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5"
Sep 06 19:32:28.831891 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:32:28.831871    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5\": container with ID starting with 86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5 not found: ID does not exist" containerID="86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5"
Sep 06 19:32:28.831942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.831902    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5"} err="failed to get container status \"86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5\": rpc error: code = NotFound desc = could not find container \"86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5\": container with ID starting with 86bbc640f547ac4d1c360d668a28412527d1c1be454dda1904b5fd1a361639e5 not found: ID does not exist"
Sep 06 19:32:28.831942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.831920    2581 scope.go:117] "RemoveContainer" containerID="c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1"
Sep 06 19:32:28.832219 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:32:28.832199    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1\": container with ID starting with c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1 not found: ID does not exist" containerID="c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1"
Sep 06 19:32:28.832272 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.832226    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1"} err="failed to get container status \"c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1\": rpc error: code = NotFound desc = could not find container \"c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1\": container with ID starting with c17fb1063171b126cb4453ee6abdfc34704932587540714020c7e17e5d85d3b1 not found: ID does not exist"
Sep 06 19:32:28.833406 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.833387    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"]
Sep 06 19:32:28.836530 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:28.836508    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-c6813a-openshift-default-9dcbd9bcb-w8gth"]
Sep 06 19:32:30.266508 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:30.266470    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" path="/var/lib/kubelet/pods/c48602bf-ce7e-4e74-9a3a-fe50b1c45d89/volumes"
Sep 06 19:32:32.013757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:32.013720    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:32.114288 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:32.114208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:32.234597 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:32.234562    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"]
Sep 06 19:32:32.234925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:32.234900    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy" containerID="cri-o://93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785" gracePeriod=30
Sep 06 19:32:32.254564 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:32.254532    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:32.574160 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:32.574120    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:32.676000 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:32.675968    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:32.795938 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:32.795901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:37.485693 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.485670    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:32:37.534020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.533930    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-credential-socket\") pod \"39f42f82-7e87-48f1-98aa-0b212de177e1\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") "
Sep 06 19:32:37.534020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.533977    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-podinfo\") pod \"39f42f82-7e87-48f1-98aa-0b212de177e1\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") "
Sep 06 19:32:37.534020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.533997    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-envoy\") pod \"39f42f82-7e87-48f1-98aa-0b212de177e1\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") "
Sep 06 19:32:37.534354 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbpq9\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-kube-api-access-cbpq9\") pod \"39f42f82-7e87-48f1-98aa-0b212de177e1\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") "
Sep 06 19:32:37.534354 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534047    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-token\") pod \"39f42f82-7e87-48f1-98aa-0b212de177e1\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") "
Sep 06 19:32:37.534354 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534134    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-data\") pod \"39f42f82-7e87-48f1-98aa-0b212de177e1\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") "
Sep 06 19:32:37.534354 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534162    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-socket\") pod \"39f42f82-7e87-48f1-98aa-0b212de177e1\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") "
Sep 06 19:32:37.534354 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534215    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-certs\") pod \"39f42f82-7e87-48f1-98aa-0b212de177e1\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") "
Sep 06 19:32:37.534354 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534279    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39f42f82-7e87-48f1-98aa-0b212de177e1-istiod-ca-cert\") pod \"39f42f82-7e87-48f1-98aa-0b212de177e1\" (UID: \"39f42f82-7e87-48f1-98aa-0b212de177e1\") "
Sep 06 19:32:37.534354 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534288    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39f42f82-7e87-48f1-98aa-0b212de177e1" (UID: "39f42f82-7e87-48f1-98aa-0b212de177e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:37.534727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534455    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39f42f82-7e87-48f1-98aa-0b212de177e1" (UID: "39f42f82-7e87-48f1-98aa-0b212de177e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:37.534727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534689    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:37.534727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534709    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:37.535051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534780    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39f42f82-7e87-48f1-98aa-0b212de177e1" (UID: "39f42f82-7e87-48f1-98aa-0b212de177e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:37.535051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.534846    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "39f42f82-7e87-48f1-98aa-0b212de177e1" (UID: "39f42f82-7e87-48f1-98aa-0b212de177e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:37.535051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.535041    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39f42f82-7e87-48f1-98aa-0b212de177e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39f42f82-7e87-48f1-98aa-0b212de177e1" (UID: "39f42f82-7e87-48f1-98aa-0b212de177e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:32:37.536558 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.536524    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-kube-api-access-cbpq9" (OuterVolumeSpecName: "kube-api-access-cbpq9") pod "39f42f82-7e87-48f1-98aa-0b212de177e1" (UID: "39f42f82-7e87-48f1-98aa-0b212de177e1"). InnerVolumeSpecName "kube-api-access-cbpq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:32:37.536683 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.536561    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39f42f82-7e87-48f1-98aa-0b212de177e1" (UID: "39f42f82-7e87-48f1-98aa-0b212de177e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:32:37.536775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.536751    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "39f42f82-7e87-48f1-98aa-0b212de177e1" (UID: "39f42f82-7e87-48f1-98aa-0b212de177e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:32:37.536835 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.536767    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39f42f82-7e87-48f1-98aa-0b212de177e1" (UID: "39f42f82-7e87-48f1-98aa-0b212de177e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:37.635645 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.635603    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:37.635645 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.635639    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:37.635645 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.635648    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:37.635645 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.635656    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39f42f82-7e87-48f1-98aa-0b212de177e1-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:37.635919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.635664    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:37.635919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.635672    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39f42f82-7e87-48f1-98aa-0b212de177e1-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:37.635919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.635679    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbpq9\" (UniqueName: \"kubernetes.io/projected/39f42f82-7e87-48f1-98aa-0b212de177e1-kube-api-access-cbpq9\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:37.877214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.877113    2581 generic.go:356] "Generic (PLEG): container finished" podID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerID="93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785" exitCode=0
Sep 06 19:32:37.877214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.877172    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"
Sep 06 19:32:37.877214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.877201    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" event={"ID":"39f42f82-7e87-48f1-98aa-0b212de177e1","Type":"ContainerDied","Data":"93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785"}
Sep 06 19:32:37.877483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.877295    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs" event={"ID":"39f42f82-7e87-48f1-98aa-0b212de177e1","Type":"ContainerDied","Data":"b347f9c740daa383b83aeba650107c2c1a8e7be53f6d0d7e9e727faedcaadc15"}
Sep 06 19:32:37.877483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.877324    2581 scope.go:117] "RemoveContainer" containerID="93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785"
Sep 06 19:32:37.887517 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.887499    2581 scope.go:117] "RemoveContainer" containerID="38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671"
Sep 06 19:32:37.897442 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.897415    2581 scope.go:117] "RemoveContainer" containerID="93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785"
Sep 06 19:32:37.897835 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:32:37.897813    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785\": container with ID starting with 93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785 not found: ID does not exist" containerID="93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785"
Sep 06 19:32:37.897925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.897846    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785"} err="failed to get container status \"93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785\": rpc error: code = NotFound desc = could not find container \"93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785\": container with ID starting with 93d73f47a63cf0a5d0c11ce79098f357ecc7143c1f687a29fa85aa53bde0e785 not found: ID does not exist"
Sep 06 19:32:37.897925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.897866    2581 scope.go:117] "RemoveContainer" containerID="38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671"
Sep 06 19:32:37.898224 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:32:37.898201    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671\": container with ID starting with 38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671 not found: ID does not exist" containerID="38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671"
Sep 06 19:32:37.898281 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.898230    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671"} err="failed to get container status \"38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671\": rpc error: code = NotFound desc = could not find container \"38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671\": container with ID starting with 38173abbe521158f093c7fa8e02941d79d6f21b2c311b6515ebdf31f992f2671 not found: ID does not exist"
Sep 06 19:32:37.899494 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.899472    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"]
Sep 06 19:32:37.903270 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:37.903245    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-032bc360-openshift-default-5b95f7ff76-pdfqs"]
Sep 06 19:32:38.267158 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:38.267129    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" path="/var/lib/kubelet/pods/39f42f82-7e87-48f1-98aa-0b212de177e1/volumes"
Sep 06 19:32:38.610292 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:38.610201    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:38.673176 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:38.673145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:38.749329 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:38.749297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:38.815244 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:38.815208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:38.933378 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:38.933342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:39.372106 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:39.372000    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:43.077161 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:43.077127    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:43.162276 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:43.162232    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"]
Sep 06 19:32:43.162905 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:43.162874    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv" podUID="8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" containerName="istio-proxy" containerID="cri-o://cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d" gracePeriod=30
Sep 06 19:32:43.176817 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:43.176789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:43.398925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:43.398873    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:43.499439 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:43.499411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:44.692828 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:44.692793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:44.748990 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:44.748958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:45.544989 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.544959    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:45.793300 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.793263    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"]
Sep 06 19:32:45.793959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.793937    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy"
Sep 06 19:32:45.794007 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.793965    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy"
Sep 06 19:32:45.794007 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.793994    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy"
Sep 06 19:32:45.794007 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.794003    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy"
Sep 06 19:32:45.794186 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.794035    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy"
Sep 06 19:32:45.794186 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.794044    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy"
Sep 06 19:32:45.794186 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.794082    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy"
Sep 06 19:32:45.794186 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.794091    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy"
Sep 06 19:32:45.794337 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.794206    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy"
Sep 06 19:32:45.794337 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.794219    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy"
Sep 06 19:32:45.794337 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.794232    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48602bf-ce7e-4e74-9a3a-fe50b1c45d89" containerName="istio-proxy"
Sep 06 19:32:45.794337 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.794246    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="39f42f82-7e87-48f1-98aa-0b212de177e1" containerName="istio-proxy"
Sep 06 19:32:45.798252 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.798200    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.800378 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.800354    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-cf729a-openshift-default-dockercfg-g87zx\""
Sep 06 19:32:45.807672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.807644    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"]
Sep 06 19:32:45.814452 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.814423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:45.816569 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.816549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-socket\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.816637 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.816597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-certs\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.816637 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.816615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-data\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.816708 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.816691    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-podinfo\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.816755 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.816744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-token\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.816834 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.816822    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n44s7\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-kube-api-access-n44s7\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.816873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.816843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8819cce4-d25e-41f7-8ef2-a0732178849f-istiod-ca-cert\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.816873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.816861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-credential-socket\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.816939 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.816882    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-envoy\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.918211 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.918154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n44s7\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-kube-api-access-n44s7\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.918424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.918217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8819cce4-d25e-41f7-8ef2-a0732178849f-istiod-ca-cert\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.918424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.918253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-credential-socket\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.918567 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.918505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-envoy\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.918630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.918600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-socket\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.918688 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.918673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-certs\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.918753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.918703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-data\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.918753 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.918734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-podinfo\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.918852 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.918793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-token\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.919486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.919230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-certs\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.919486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.919300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8819cce4-d25e-41f7-8ef2-a0732178849f-istiod-ca-cert\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.919486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.919481    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-credential-socket\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.919745 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.919578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-socket\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.919996 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.919972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-data\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.922156 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.922128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-podinfo\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.922255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.922131    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-envoy\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.926217 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.926197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-token\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:45.926302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:45.926256    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n44s7\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-kube-api-access-n44s7\") pod \"e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") " pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:46.040269 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:46.040236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:46.111400 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:46.111309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:46.395481 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:46.395448    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"]
Sep 06 19:32:46.396429 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:32:46.396403    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8819cce4_d25e_41f7_8ef2_a0732178849f.slice/crio-9bb013688ee12aa7fb97d1908adaa22977dddf421304ece02995e68f422d171e WatchSource:0}: Error finding container 9bb013688ee12aa7fb97d1908adaa22977dddf421304ece02995e68f422d171e: Status 404 returned error can't find the container with id 9bb013688ee12aa7fb97d1908adaa22977dddf421304ece02995e68f422d171e
Sep 06 19:32:46.399137 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:46.399106    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:32:46.399212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:46.399181    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:32:46.399252 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:46.399210    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:32:46.936420 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:46.936386    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" event={"ID":"8819cce4-d25e-41f7-8ef2-a0732178849f","Type":"ContainerStarted","Data":"05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc"}
Sep 06 19:32:46.936767 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:46.936451    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" event={"ID":"8819cce4-d25e-41f7-8ef2-a0732178849f","Type":"ContainerStarted","Data":"9bb013688ee12aa7fb97d1908adaa22977dddf421304ece02995e68f422d171e"}
Sep 06 19:32:46.954303 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:46.954246    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" podStartSLOduration=1.954224468 podStartE2EDuration="1.954224468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:32:46.95354125 +0000 UTC m=+1295.233994043" watchObservedRunningTime="2026-09-06 19:32:46.954224468 +0000 UTC m=+1295.234677261"
Sep 06 19:32:47.111784 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:47.111745    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:47.117170 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:47.117146    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:47.943765 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:47.943731    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:47.944637 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:47.944620    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:32:48.434958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.434936    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:48.509116 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.509078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:48.546955 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.546861    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-envoy\") pod \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") "
Sep 06 19:32:48.546955 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.546912    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-token\") pod \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") "
Sep 06 19:32:48.547230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.546962    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-certs\") pod \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") "
Sep 06 19:32:48.547230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.546989    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istiod-ca-cert\") pod \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") "
Sep 06 19:32:48.547230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547054    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-podinfo\") pod \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") "
Sep 06 19:32:48.547230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547156    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksxrg\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-kube-api-access-ksxrg\") pod \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") "
Sep 06 19:32:48.547230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547187    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-credential-socket\") pod \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") "
Sep 06 19:32:48.547230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547214    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-data\") pod \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") "
Sep 06 19:32:48.547552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547249    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-socket\") pod \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\" (UID: \"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b\") "
Sep 06 19:32:48.547552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547304    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" (UID: "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:48.547552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547499    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" (UID: "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:48.547552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547493    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" (UID: "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:32:48.547925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547685    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:48.547925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547691    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" (UID: "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:48.547925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547701    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-data" (OuterVolumeSpecName: "istio-data") pod "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" (UID: "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:48.547925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547707    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:48.547925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.547723    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:48.549499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.549468    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" (UID: "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:32:48.549499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.549480    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-token" (OuterVolumeSpecName: "istio-token") pod "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" (UID: "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:32:48.549750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.549726    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" (UID: "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:32:48.549807 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.549790    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-kube-api-access-ksxrg" (OuterVolumeSpecName: "kube-api-access-ksxrg") pod "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" (UID: "8bfd79b5-3c1a-4f2f-b2bf-ab671194351b"). InnerVolumeSpecName "kube-api-access-ksxrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:32:48.648885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.648836    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:48.648885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.648871    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:48.648885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.648880    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:48.648885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.648889    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksxrg\" (UniqueName: \"kubernetes.io/projected/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-kube-api-access-ksxrg\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:48.648885 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.648899    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:48.649241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.648908    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:48.800329 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.800247    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:48.889275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.889235    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:48.951657 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.951621    2581 generic.go:356] "Generic (PLEG): container finished" podID="8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" containerID="cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d" exitCode=0
Sep 06 19:32:48.952173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.951677    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"
Sep 06 19:32:48.952173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.951683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv" event={"ID":"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b","Type":"ContainerDied","Data":"cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d"}
Sep 06 19:32:48.952173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.951756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv" event={"ID":"8bfd79b5-3c1a-4f2f-b2bf-ab671194351b","Type":"ContainerDied","Data":"587835b06de64d4e1eb2c24af2d7b4398ffebce82fd656ef17a30d028bcdf75e"}
Sep 06 19:32:48.952173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.951790    2581 scope.go:117] "RemoveContainer" containerID="cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d"
Sep 06 19:32:48.961666 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.961648    2581 scope.go:117] "RemoveContainer" containerID="cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d"
Sep 06 19:32:48.961925 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:32:48.961907    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d\": container with ID starting with cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d not found: ID does not exist" containerID="cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d"
Sep 06 19:32:48.961970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.961933    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d"} err="failed to get container status \"cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d\": rpc error: code = NotFound desc = could not find container \"cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d\": container with ID starting with cf2e5b7280ff8c3c1e2e1f27756e66c573926d37539fc9790d353f5d9de0250d not found: ID does not exist"
Sep 06 19:32:48.973640 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.973609    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"]
Sep 06 19:32:48.975678 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:48.975655    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-477eea-openshift-default-86994c9b8b-dmvhv"]
Sep 06 19:32:49.982360 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:49.982306    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:50.083368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:50.083338    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:50.225468 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:50.225421    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:50.267895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:50.267815    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" path="/var/lib/kubelet/pods/8bfd79b5-3c1a-4f2f-b2bf-ab671194351b/volumes"
Sep 06 19:32:50.414545 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:50.414513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:50.522410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:50.522320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:51.925382 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:51.925349    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"]
Sep 06 19:32:51.941029 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:51.941000    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:52.398391 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:52.398357    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:53.336762 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.336731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:32:53.438430 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.438388    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"]
Sep 06 19:32:53.438828 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.438816    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" containerName="istio-proxy"
Sep 06 19:32:53.438887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.438831    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" containerName="istio-proxy"
Sep 06 19:32:53.438924 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.438911    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bfd79b5-3c1a-4f2f-b2bf-ab671194351b" containerName="istio-proxy"
Sep 06 19:32:53.443617 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.443597    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"
Sep 06 19:32:53.446042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.445831    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:32:53.446042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.445908    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:32:53.448085 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.448035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"]
Sep 06 19:32:53.496890 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.496846    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph\" (UID: \"02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"
Sep 06 19:32:53.597804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.597718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph\" (UID: \"02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"
Sep 06 19:32:53.598377 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.598358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph\" (UID: \"02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"
Sep 06 19:32:53.756891 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:53.756858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"
Sep 06 19:32:54.029244 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:54.029218    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"]
Sep 06 19:32:54.031346 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:32:54.031319    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c8c3c3_0502_4d9c_9dd7_8730b9bff7b0.slice/crio-463c8093c574196c4707e17b424b07f59006509a3285f69f0b0b42a9ce1aed73 WatchSource:0}: Error finding container 463c8093c574196c4707e17b424b07f59006509a3285f69f0b0b42a9ce1aed73: Status 404 returned error can't find the container with id 463c8093c574196c4707e17b424b07f59006509a3285f69f0b0b42a9ce1aed73
Sep 06 19:32:54.990750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:54.990665    2581 generic.go:356] "Generic (PLEG): container finished" podID="02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0" containerID="7c5e6039f3ede8b195b56af177d2c3066c8c5e4c02939ab8afc61fbab30fc9e5" exitCode=0
Sep 06 19:32:54.990750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:54.990712    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph" event={"ID":"02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0","Type":"ContainerDied","Data":"7c5e6039f3ede8b195b56af177d2c3066c8c5e4c02939ab8afc61fbab30fc9e5"}
Sep 06 19:32:54.990750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:54.990750    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph" event={"ID":"02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0","Type":"ContainerStarted","Data":"463c8093c574196c4707e17b424b07f59006509a3285f69f0b0b42a9ce1aed73"}
Sep 06 19:32:56.141208 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:56.141179    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"
Sep 06 19:32:56.224375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:56.224337    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0-maas-api-service-ca\") pod \"02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0\" (UID: \"02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0\") "
Sep 06 19:32:56.224714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:56.224687    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0" (UID: "02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:32:56.325827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:56.325785    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:32:57.003609 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:57.003580    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"
Sep 06 19:32:57.003609 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:57.003614    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph" event={"ID":"02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0","Type":"ContainerDied","Data":"463c8093c574196c4707e17b424b07f59006509a3285f69f0b0b42a9ce1aed73"}
Sep 06 19:32:57.003833 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:32:57.003658    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="463c8093c574196c4707e17b424b07f59006509a3285f69f0b0b42a9ce1aed73"
Sep 06 19:33:03.824647 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:03.824612    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:03.923961 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:03.923916    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:03.995183 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:03.995116    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:04.008103 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:04.008052    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"]
Sep 06 19:33:04.014902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:04.014851    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5307a7-4w2ph"]
Sep 06 19:33:04.046894 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:04.046847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:04.266549 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:04.266514    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0" path="/var/lib/kubelet/pods/02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0/volumes"
Sep 06 19:33:04.356977 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:04.356945    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:06.950225 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:06.950177    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr" podUID="5b14c50e-49fa-404a-939e-0f74f31ae229" containerName="main" containerID="cri-o://d0ce754c05a53ada2eb1f4fef2ccb4f803a5b9fc86bd12fd93726991b4a5aa19" gracePeriod=45
Sep 06 19:33:07.073594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.073553    2581 generic.go:356] "Generic (PLEG): container finished" podID="5b14c50e-49fa-404a-939e-0f74f31ae229" containerID="d0ce754c05a53ada2eb1f4fef2ccb4f803a5b9fc86bd12fd93726991b4a5aa19" exitCode=0
Sep 06 19:33:07.073726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.073621    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr" event={"ID":"5b14c50e-49fa-404a-939e-0f74f31ae229","Type":"ContainerDied","Data":"d0ce754c05a53ada2eb1f4fef2ccb4f803a5b9fc86bd12fd93726991b4a5aa19"}
Sep 06 19:33:07.213523 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.213441    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:33:07.349333 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349287    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-model-cache\") pod \"5b14c50e-49fa-404a-939e-0f74f31ae229\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") "
Sep 06 19:33:07.349588 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349349    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-tmp-dir\") pod \"5b14c50e-49fa-404a-939e-0f74f31ae229\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") "
Sep 06 19:33:07.349588 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349393    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b14c50e-49fa-404a-939e-0f74f31ae229-tls-certs\") pod \"5b14c50e-49fa-404a-939e-0f74f31ae229\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") "
Sep 06 19:33:07.349588 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349456    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh5px\" (UniqueName: \"kubernetes.io/projected/5b14c50e-49fa-404a-939e-0f74f31ae229-kube-api-access-zh5px\") pod \"5b14c50e-49fa-404a-939e-0f74f31ae229\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") "
Sep 06 19:33:07.349588 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349484    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-home\") pod \"5b14c50e-49fa-404a-939e-0f74f31ae229\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") "
Sep 06 19:33:07.349588 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-dshm\") pod \"5b14c50e-49fa-404a-939e-0f74f31ae229\" (UID: \"5b14c50e-49fa-404a-939e-0f74f31ae229\") "
Sep 06 19:33:07.349887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349621    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-model-cache" (OuterVolumeSpecName: "model-cache") pod "5b14c50e-49fa-404a-939e-0f74f31ae229" (UID: "5b14c50e-49fa-404a-939e-0f74f31ae229"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:07.349887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349629    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5b14c50e-49fa-404a-939e-0f74f31ae229" (UID: "5b14c50e-49fa-404a-939e-0f74f31ae229"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:07.349887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349783    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-home" (OuterVolumeSpecName: "home") pod "5b14c50e-49fa-404a-939e-0f74f31ae229" (UID: "5b14c50e-49fa-404a-939e-0f74f31ae229"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:07.349887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349852    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-home\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:07.349887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349868    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-model-cache\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:07.349887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.349886    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-tmp-dir\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:07.352000 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.351973    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-dshm" (OuterVolumeSpecName: "dshm") pod "5b14c50e-49fa-404a-939e-0f74f31ae229" (UID: "5b14c50e-49fa-404a-939e-0f74f31ae229"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:07.352000 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.351984    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b14c50e-49fa-404a-939e-0f74f31ae229-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5b14c50e-49fa-404a-939e-0f74f31ae229" (UID: "5b14c50e-49fa-404a-939e-0f74f31ae229"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:33:07.352160 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.352038    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b14c50e-49fa-404a-939e-0f74f31ae229-kube-api-access-zh5px" (OuterVolumeSpecName: "kube-api-access-zh5px") pod "5b14c50e-49fa-404a-939e-0f74f31ae229" (UID: "5b14c50e-49fa-404a-939e-0f74f31ae229"). InnerVolumeSpecName "kube-api-access-zh5px". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:33:07.451196 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.451157    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b14c50e-49fa-404a-939e-0f74f31ae229-dshm\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:07.451196 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.451187    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b14c50e-49fa-404a-939e-0f74f31ae229-tls-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:07.451196 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:07.451197    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh5px\" (UniqueName: \"kubernetes.io/projected/5b14c50e-49fa-404a-939e-0f74f31ae229-kube-api-access-zh5px\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:08.081419 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.081379    2581 generic.go:356] "Generic (PLEG): container finished" podID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerID="05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc" exitCode=0
Sep 06 19:33:08.081890 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.081456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" event={"ID":"8819cce4-d25e-41f7-8ef2-a0732178849f","Type":"ContainerDied","Data":"05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc"}
Sep 06 19:33:08.081968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.081951    2581 scope.go:117] "RemoveContainer" containerID="05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc"
Sep 06 19:33:08.083011 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.082992    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr" event={"ID":"5b14c50e-49fa-404a-939e-0f74f31ae229","Type":"ContainerDied","Data":"41c396455540a417546cec882b4540696f6c2126fc546c7df725461e48332973"}
Sep 06 19:33:08.083118 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.083027    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"
Sep 06 19:33:08.083118 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.083037    2581 scope.go:117] "RemoveContainer" containerID="d0ce754c05a53ada2eb1f4fef2ccb4f803a5b9fc86bd12fd93726991b4a5aa19"
Sep 06 19:33:08.083233 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.083208    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:08.083305 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.083286    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:08.083362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.083330    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:08.117026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.116989    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"]
Sep 06 19:33:08.122908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.122874    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-5307a7/test-model-5307a7-kserve-59896ccc87-pdxzr"]
Sep 06 19:33:08.267157 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:08.267043    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b14c50e-49fa-404a-939e-0f74f31ae229" path="/var/lib/kubelet/pods/5b14c50e-49fa-404a-939e-0f74f31ae229/volumes"
Sep 06 19:33:09.094779 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:09.094730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" event={"ID":"8819cce4-d25e-41f7-8ef2-a0732178849f","Type":"ContainerStarted","Data":"7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273"}
Sep 06 19:33:09.113534 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:09.113494    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:33:09.113882 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:09.113847    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 06 19:33:09.114018 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:09.113908    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 06 19:33:09.999875 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:09.999828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:10.099986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.099951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:10.112475 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.112440    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 06 19:33:10.112684 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.112522    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 06 19:33:10.227714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.227680    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"]
Sep 06 19:33:10.227939 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.227911    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy" containerID="cri-o://8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6" gracePeriod=30
Sep 06 19:33:10.257719 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.257627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:10.561408 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.561315    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"]
Sep 06 19:33:10.561797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.561782    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b14c50e-49fa-404a-939e-0f74f31ae229" containerName="main"
Sep 06 19:33:10.561840 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.561799    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b14c50e-49fa-404a-939e-0f74f31ae229" containerName="main"
Sep 06 19:33:10.561840 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.561820    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0" containerName="revoke-keys"
Sep 06 19:33:10.561840 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.561826    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0" containerName="revoke-keys"
Sep 06 19:33:10.561933 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.561902    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b14c50e-49fa-404a-939e-0f74f31ae229" containerName="main"
Sep 06 19:33:10.561933 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.561914    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="02c8c3c3-0502-4d9c-9dd7-8730b9bff7b0" containerName="revoke-keys"
Sep 06 19:33:10.565039 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.565023    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"
Sep 06 19:33:10.566974 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.566955    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:33:10.567096 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.567054    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:33:10.571961 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.571892    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"]
Sep 06 19:33:10.582141 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.582115    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c730b4d-efd1-4799-8cd6-5c96890f9bbf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c\" (UID: \"0c730b4d-efd1-4799-8cd6-5c96890f9bbf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"
Sep 06 19:33:10.683137 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.683098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c730b4d-efd1-4799-8cd6-5c96890f9bbf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c\" (UID: \"0c730b4d-efd1-4799-8cd6-5c96890f9bbf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"
Sep 06 19:33:10.683700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.683681    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c730b4d-efd1-4799-8cd6-5c96890f9bbf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c\" (UID: \"0c730b4d-efd1-4799-8cd6-5c96890f9bbf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"
Sep 06 19:33:10.790695 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.790604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:10.879169 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:10.879055    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"
Sep 06 19:33:11.112034 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:11.111990    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 06 19:33:11.112628 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:11.112053    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 06 19:33:11.173379 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:11.173350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"]
Sep 06 19:33:11.175346 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:33:11.175316    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c730b4d_efd1_4799_8cd6_5c96890f9bbf.slice/crio-eb169ae6f693a5005a49e6eec7565468f7b036d09435022899074a834cec7331 WatchSource:0}: Error finding container eb169ae6f693a5005a49e6eec7565468f7b036d09435022899074a834cec7331: Status 404 returned error can't find the container with id eb169ae6f693a5005a49e6eec7565468f7b036d09435022899074a834cec7331
Sep 06 19:33:11.767012 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:11.766978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:12.112487 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:12.112446    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 06 19:33:12.112891 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:12.112518    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 06 19:33:12.122315 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:12.122285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c" event={"ID":"0c730b4d-efd1-4799-8cd6-5c96890f9bbf","Type":"ContainerStarted","Data":"a8357ae7f25d52ae9a827f5fa8a2d6aa40ebddfa4c2e9a03419034340a4d3e06"}
Sep 06 19:33:12.122536 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:12.122343    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c" event={"ID":"0c730b4d-efd1-4799-8cd6-5c96890f9bbf","Type":"ContainerStarted","Data":"eb169ae6f693a5005a49e6eec7565468f7b036d09435022899074a834cec7331"}
Sep 06 19:33:12.136006 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:12.135953    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c" podStartSLOduration=2.135937558 podStartE2EDuration="2.135937558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:33:12.134317907 +0000 UTC m=+1320.414770702" watchObservedRunningTime="2026-09-06 19:33:12.135937558 +0000 UTC m=+1320.416390351"
Sep 06 19:33:13.112758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:13.112716    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 06 19:33:13.113291 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:13.112805    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 06 19:33:13.130224 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:13.130185    2581 generic.go:356] "Generic (PLEG): container finished" podID="0c730b4d-efd1-4799-8cd6-5c96890f9bbf" containerID="a8357ae7f25d52ae9a827f5fa8a2d6aa40ebddfa4c2e9a03419034340a4d3e06" exitCode=0
Sep 06 19:33:13.130411 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:13.130265    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c" event={"ID":"0c730b4d-efd1-4799-8cd6-5c96890f9bbf","Type":"ContainerDied","Data":"a8357ae7f25d52ae9a827f5fa8a2d6aa40ebddfa4c2e9a03419034340a4d3e06"}
Sep 06 19:33:13.705392 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:13.705345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:14.077386 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.077299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:14.112759 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.112719    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 06 19:33:14.113239 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.112793    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 06 19:33:14.316942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.316835    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:14.320747 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.320709    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"]
Sep 06 19:33:14.325972 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.325945    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"]
Sep 06 19:33:14.326133 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.326076    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.328200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.328131    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-6bbb53-openshift-default-dockercfg-qb5jd\""
Sep 06 19:33:14.334694 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.334670    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"
Sep 06 19:33:14.445403 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.445369    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c730b4d-efd1-4799-8cd6-5c96890f9bbf-maas-api-service-ca\") pod \"0c730b4d-efd1-4799-8cd6-5c96890f9bbf\" (UID: \"0c730b4d-efd1-4799-8cd6-5c96890f9bbf\") "
Sep 06 19:33:14.445594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.445561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-data\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.445594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.445591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-istio-token\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.445701 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.445624    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-socket\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.445701 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.445650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883854de-8b63-45da-9763-f73dea0cb887-istiod-ca-cert\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.445806 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.445785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c730b4d-efd1-4799-8cd6-5c96890f9bbf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c730b4d-efd1-4799-8cd6-5c96890f9bbf" (UID: "0c730b4d-efd1-4799-8cd6-5c96890f9bbf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:33:14.445862 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.445830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-envoy\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.445968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.445918    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-credential-socket\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.446046 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.445989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv5xq\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-kube-api-access-dv5xq\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.446046 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.446034    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883854de-8b63-45da-9763-f73dea0cb887-istio-podinfo\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.446206 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.446107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-certs\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.446258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.446204    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c730b4d-efd1-4799-8cd6-5c96890f9bbf-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:14.547756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.547701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-socket\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.547756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.547756    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883854de-8b63-45da-9763-f73dea0cb887-istiod-ca-cert\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.547812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-envoy\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.547871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-credential-socket\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.547928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv5xq\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-kube-api-access-dv5xq\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.547964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883854de-8b63-45da-9763-f73dea0cb887-istio-podinfo\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.547995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-certs\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548344 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.548041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-data\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548344 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.548097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-istio-token\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.548356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-credential-socket\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548609 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.548584    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-socket\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.548611    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-certs\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.548687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.548627    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-data\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.549052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.549028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883854de-8b63-45da-9763-f73dea0cb887-istiod-ca-cert\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.550733 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.550667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-envoy\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.551249 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.551220    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883854de-8b63-45da-9763-f73dea0cb887-istio-podinfo\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.556591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.556548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-istio-token\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.556591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.556571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv5xq\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-kube-api-access-dv5xq\") pod \"e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") " pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.598931 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.598847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:14.641492 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.641453    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:14.933294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.933266    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"]
Sep 06 19:33:14.935816 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:33:14.935777    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883854de_8b63_45da_9763_f73dea0cb887.slice/crio-edb9d666050ecc52f7e74f30ba022270d7b459170023569197454bd14b4bf868 WatchSource:0}: Error finding container edb9d666050ecc52f7e74f30ba022270d7b459170023569197454bd14b4bf868: Status 404 returned error can't find the container with id edb9d666050ecc52f7e74f30ba022270d7b459170023569197454bd14b4bf868
Sep 06 19:33:14.938215 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.938183    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:14.938283 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.938256    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:14.938326 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:14.938285    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:15.116929 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.116892    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:33:15.117460 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.117288    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:33:15.118085 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.118045    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:33:15.144420 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.144391    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"
Sep 06 19:33:15.144629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.144428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c" event={"ID":"0c730b4d-efd1-4799-8cd6-5c96890f9bbf","Type":"ContainerDied","Data":"eb169ae6f693a5005a49e6eec7565468f7b036d09435022899074a834cec7331"}
Sep 06 19:33:15.144629 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.144484    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb169ae6f693a5005a49e6eec7565468f7b036d09435022899074a834cec7331"
Sep 06 19:33:15.146295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.146264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" event={"ID":"883854de-8b63-45da-9763-f73dea0cb887","Type":"ContainerStarted","Data":"5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb"}
Sep 06 19:33:15.146453 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.146324    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" event={"ID":"883854de-8b63-45da-9763-f73dea0cb887","Type":"ContainerStarted","Data":"edb9d666050ecc52f7e74f30ba022270d7b459170023569197454bd14b4bf868"}
Sep 06 19:33:15.178726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.178678    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" podStartSLOduration=1.178659599 podStartE2EDuration="1.178659599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:33:15.173121677 +0000 UTC m=+1323.453574473" watchObservedRunningTime="2026-09-06 19:33:15.178659599 +0000 UTC m=+1323.459112393"
Sep 06 19:33:15.517430 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.517402    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:33:15.560043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560012    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e562473-4e05-4539-8d95-b4c0a7439b13-istiod-ca-cert\") pod \"6e562473-4e05-4539-8d95-b4c0a7439b13\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") "
Sep 06 19:33:15.560043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560054    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-podinfo\") pod \"6e562473-4e05-4539-8d95-b4c0a7439b13\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") "
Sep 06 19:33:15.560301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560121    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-socket\") pod \"6e562473-4e05-4539-8d95-b4c0a7439b13\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") "
Sep 06 19:33:15.560301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560145    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f66pc\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-kube-api-access-f66pc\") pod \"6e562473-4e05-4539-8d95-b4c0a7439b13\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") "
Sep 06 19:33:15.560301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560162    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-data\") pod \"6e562473-4e05-4539-8d95-b4c0a7439b13\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") "
Sep 06 19:33:15.560301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560209    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-envoy\") pod \"6e562473-4e05-4539-8d95-b4c0a7439b13\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") "
Sep 06 19:33:15.560301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560243    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-certs\") pod \"6e562473-4e05-4539-8d95-b4c0a7439b13\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") "
Sep 06 19:33:15.560301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560273    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-credential-socket\") pod \"6e562473-4e05-4539-8d95-b4c0a7439b13\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") "
Sep 06 19:33:15.560601 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560348    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-token\") pod \"6e562473-4e05-4539-8d95-b4c0a7439b13\" (UID: \"6e562473-4e05-4539-8d95-b4c0a7439b13\") "
Sep 06 19:33:15.560601 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560455    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e562473-4e05-4539-8d95-b4c0a7439b13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e562473-4e05-4539-8d95-b4c0a7439b13" (UID: "6e562473-4e05-4539-8d95-b4c0a7439b13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:33:15.560601 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560506    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e562473-4e05-4539-8d95-b4c0a7439b13" (UID: "6e562473-4e05-4539-8d95-b4c0a7439b13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:15.560601 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560582    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e562473-4e05-4539-8d95-b4c0a7439b13" (UID: "6e562473-4e05-4539-8d95-b4c0a7439b13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:15.560821 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560663    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e562473-4e05-4539-8d95-b4c0a7439b13" (UID: "6e562473-4e05-4539-8d95-b4c0a7439b13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:15.560821 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560698    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e562473-4e05-4539-8d95-b4c0a7439b13" (UID: "6e562473-4e05-4539-8d95-b4c0a7439b13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:15.560908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560830    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e562473-4e05-4539-8d95-b4c0a7439b13-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:15.560908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560850    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:15.560908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560870    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:15.560908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560885    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:15.560908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.560897    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:15.562811 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.562782    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-kube-api-access-f66pc" (OuterVolumeSpecName: "kube-api-access-f66pc") pod "6e562473-4e05-4539-8d95-b4c0a7439b13" (UID: "6e562473-4e05-4539-8d95-b4c0a7439b13"). InnerVolumeSpecName "kube-api-access-f66pc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:33:15.562944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.562829    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e562473-4e05-4539-8d95-b4c0a7439b13" (UID: "6e562473-4e05-4539-8d95-b4c0a7439b13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:33:15.562944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.562905    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e562473-4e05-4539-8d95-b4c0a7439b13" (UID: "6e562473-4e05-4539-8d95-b4c0a7439b13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:15.563051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.563028    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e562473-4e05-4539-8d95-b4c0a7439b13" (UID: "6e562473-4e05-4539-8d95-b4c0a7439b13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:33:15.641892 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.641851    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:15.647102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.647043    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:15.661775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.661741    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f66pc\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-kube-api-access-f66pc\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:15.661775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.661773    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:15.661975 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.661789    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:15.661975 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:15.661802    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e562473-4e05-4539-8d95-b4c0a7439b13-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:16.154803 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.154764    2581 generic.go:356] "Generic (PLEG): container finished" podID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerID="8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6" exitCode=0
Sep 06 19:33:16.155289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.154828    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"
Sep 06 19:33:16.155289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.154842    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" event={"ID":"6e562473-4e05-4539-8d95-b4c0a7439b13","Type":"ContainerDied","Data":"8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6"}
Sep 06 19:33:16.155289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.154912    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh" event={"ID":"6e562473-4e05-4539-8d95-b4c0a7439b13","Type":"ContainerDied","Data":"50ff8ed0daa9dede55e00109931e024dc17f227a4f9ea466df437d2ad1d1ce5a"}
Sep 06 19:33:16.155289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.154934    2581 scope.go:117] "RemoveContainer" containerID="8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6"
Sep 06 19:33:16.155567 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.155548    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:16.156623 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.156599    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:16.165504 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.165484    2581 scope.go:117] "RemoveContainer" containerID="53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d"
Sep 06 19:33:16.175178 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.175158    2581 scope.go:117] "RemoveContainer" containerID="8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6"
Sep 06 19:33:16.175732 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:33:16.175543    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6\": container with ID starting with 8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6 not found: ID does not exist" containerID="8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6"
Sep 06 19:33:16.175732 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.175613    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6"} err="failed to get container status \"8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6\": rpc error: code = NotFound desc = could not find container \"8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6\": container with ID starting with 8af0dfc00618df3ad0f6dbe97942fc29d2292699754a1490d9087a9975c77eb6 not found: ID does not exist"
Sep 06 19:33:16.175732 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.175662    2581 scope.go:117] "RemoveContainer" containerID="53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d"
Sep 06 19:33:16.176023 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:33:16.176002    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d\": container with ID starting with 53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d not found: ID does not exist" containerID="53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d"
Sep 06 19:33:16.176102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.176028    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d"} err="failed to get container status \"53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d\": rpc error: code = NotFound desc = could not find container \"53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d\": container with ID starting with 53abfa67d2dac6d862e8a3cbb564dba6756e7f59f2b1b572eeeacb6aa12bd63d not found: ID does not exist"
Sep 06 19:33:16.196141 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.194773    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"]
Sep 06 19:33:16.198410 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.198366    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-5307a7-openshift-default-6dfc457456-bx2xh"]
Sep 06 19:33:16.266823 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:16.266782    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" path="/var/lib/kubelet/pods/6e562473-4e05-4539-8d95-b4c0a7439b13/volumes"
Sep 06 19:33:17.023568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:17.023537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:17.118674 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:17.118639    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:18.180092 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:18.180035    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:18.246466 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:18.246434    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:18.781203 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:18.781169    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:18.873553 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:18.873513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:19.093335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:19.093245    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:20.685541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:20.685502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:21.197868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:21.197830    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:21.296464 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:21.296411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:21.367862 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:21.367812    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"]
Sep 06 19:33:21.369483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:21.369440    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:21.370075 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:21.370041    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d73678-m4z6c"]
Sep 06 19:33:21.420226 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:21.420194    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:21.700695 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:21.700664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:22.271773 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:22.271733    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c730b4d-efd1-4799-8cd6-5c96890f9bbf" path="/var/lib/kubelet/pods/0c730b4d-efd1-4799-8cd6-5c96890f9bbf/volumes"
Sep 06 19:33:25.392873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:25.392834    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:27.489319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:27.489277    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:27.590292 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:27.590258    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:27.723092 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:27.722514    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"]
Sep 06 19:33:27.729902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:27.729833    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:28.233620 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.233559    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy" containerID="cri-o://f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec" gracePeriod=30
Sep 06 19:33:28.260282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.260213    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:28.525876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.525791    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"]
Sep 06 19:33:28.526556 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.526537    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy"
Sep 06 19:33:28.526608 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.526560    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy"
Sep 06 19:33:28.526608 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.526587    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c730b4d-efd1-4799-8cd6-5c96890f9bbf" containerName="revoke-keys"
Sep 06 19:33:28.526608 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.526597    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c730b4d-efd1-4799-8cd6-5c96890f9bbf" containerName="revoke-keys"
Sep 06 19:33:28.526699 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.526617    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy"
Sep 06 19:33:28.526699 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.526626    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy"
Sep 06 19:33:28.526770 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.526758    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy"
Sep 06 19:33:28.526806 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.526777    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c730b4d-efd1-4799-8cd6-5c96890f9bbf" containerName="revoke-keys"
Sep 06 19:33:28.531828 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.531805    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.533972 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.533948    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-e5ef25-openshift-default-dockercfg-mdxhw\""
Sep 06 19:33:28.541458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.541433    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"]
Sep 06 19:33:28.558238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.558200    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:28.712662 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.712622    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istiod-ca-cert\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.712662 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.712663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-podinfo\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.712662 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.712687    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-socket\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.713019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.712707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-certs\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.713019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.712798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zxvj\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-kube-api-access-8zxvj\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.713019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.712837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-data\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.713019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.712907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-credential-socket\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.713019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.712943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-token\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.713019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.712976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-envoy\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814025 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.813928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istiod-ca-cert\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814025 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.813973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-podinfo\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814025 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.813996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-socket\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-certs\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zxvj\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-kube-api-access-8zxvj\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-data\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-credential-socket\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-token\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-envoy\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814430    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-socket\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-certs\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814735 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-credential-socket\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814808 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istiod-ca-cert\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.814808 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.814785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-data\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.816749 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.816724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-envoy\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.816879 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.816861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-podinfo\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.821514 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.821489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-token\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.821638 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.821622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zxvj\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-kube-api-access-8zxvj\") pod \"e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") " pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.844727 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.844689    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:28.938622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:28.938574    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:29.149100 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:29.149053    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"]
Sep 06 19:33:29.150772 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:33:29.150744    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae671dd5_c6ae_4bc1_8587_1f4906ec3f42.slice/crio-066a72c00be288969abbab7759e2cd50c344da3ccbc274328bb456d2b6f46189 WatchSource:0}: Error finding container 066a72c00be288969abbab7759e2cd50c344da3ccbc274328bb456d2b6f46189: Status 404 returned error can't find the container with id 066a72c00be288969abbab7759e2cd50c344da3ccbc274328bb456d2b6f46189
Sep 06 19:33:29.153389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:29.153340    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:29.153490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:29.153439    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:29.153536 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:29.153496    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:29.241462 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:29.241430    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" event={"ID":"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42","Type":"ContainerStarted","Data":"3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3"}
Sep 06 19:33:29.241615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:29.241495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" event={"ID":"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42","Type":"ContainerStarted","Data":"066a72c00be288969abbab7759e2cd50c344da3ccbc274328bb456d2b6f46189"}
Sep 06 19:33:29.259573 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:29.259268    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" podStartSLOduration=1.2592448059999999 podStartE2EDuration="1.259244806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:33:29.257861702 +0000 UTC m=+1337.538314544" watchObservedRunningTime="2026-09-06 19:33:29.259244806 +0000 UTC m=+1337.539697601"
Sep 06 19:33:29.844952 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:29.844914    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:30.850626 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:30.850590    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:31.114760 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:31.114622    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:31.208548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:31.208512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:31.254888 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:31.254859    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:31.255854 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:31.255833    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:32.197276 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:32.197231    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 19:33:32.197680 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:32.197309    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 19:33:32.257083 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:32.257030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:32.361609 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:32.361566    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:32.461220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:32.461135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:32.729332 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:32.729241    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:32.839876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:32.839844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:33.498577 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.498508    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:33:33.667986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.667951    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-podinfo\") pod \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") "
Sep 06 19:33:33.668218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668000    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-envoy\") pod \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") "
Sep 06 19:33:33.668218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668020    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-data\") pod \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") "
Sep 06 19:33:33.668218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668035    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istiod-ca-cert\") pod \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") "
Sep 06 19:33:33.668218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668055    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-certs\") pod \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") "
Sep 06 19:33:33.668218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668142    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw4cz\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-kube-api-access-pw4cz\") pod \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") "
Sep 06 19:33:33.668218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668187    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-socket\") pod \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") "
Sep 06 19:33:33.668547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668249    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-credential-socket\") pod \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") "
Sep 06 19:33:33.668547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668272    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-token\") pod \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\" (UID: \"2206bafb-7e52-48ec-9bb8-d661890a3a9f\") "
Sep 06 19:33:33.668547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668422    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-data" (OuterVolumeSpecName: "istio-data") pod "2206bafb-7e52-48ec-9bb8-d661890a3a9f" (UID: "2206bafb-7e52-48ec-9bb8-d661890a3a9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:33.668547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668460    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2206bafb-7e52-48ec-9bb8-d661890a3a9f" (UID: "2206bafb-7e52-48ec-9bb8-d661890a3a9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:33:33.668547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668476    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2206bafb-7e52-48ec-9bb8-d661890a3a9f" (UID: "2206bafb-7e52-48ec-9bb8-d661890a3a9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:33.668824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668648    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2206bafb-7e52-48ec-9bb8-d661890a3a9f" (UID: "2206bafb-7e52-48ec-9bb8-d661890a3a9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:33.668824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668661    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2206bafb-7e52-48ec-9bb8-d661890a3a9f" (UID: "2206bafb-7e52-48ec-9bb8-d661890a3a9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:33.668824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668763    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:33.668824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668783    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:33.668824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.668791    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:33.670491 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.670461    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2206bafb-7e52-48ec-9bb8-d661890a3a9f" (UID: "2206bafb-7e52-48ec-9bb8-d661890a3a9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:33:33.670648 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.670632    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-token" (OuterVolumeSpecName: "istio-token") pod "2206bafb-7e52-48ec-9bb8-d661890a3a9f" (UID: "2206bafb-7e52-48ec-9bb8-d661890a3a9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:33:33.670942 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.670916    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2206bafb-7e52-48ec-9bb8-d661890a3a9f" (UID: "2206bafb-7e52-48ec-9bb8-d661890a3a9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:33:33.671033 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.670977    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-kube-api-access-pw4cz" (OuterVolumeSpecName: "kube-api-access-pw4cz") pod "2206bafb-7e52-48ec-9bb8-d661890a3a9f" (UID: "2206bafb-7e52-48ec-9bb8-d661890a3a9f"). InnerVolumeSpecName "kube-api-access-pw4cz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:33:33.770033 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.769943    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:33.770033 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.769976    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:33.770033 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.769990    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw4cz\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-kube-api-access-pw4cz\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:33.770033 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.770005    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:33.770033 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.770015    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2206bafb-7e52-48ec-9bb8-d661890a3a9f-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:33.770033 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:33.770023    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2206bafb-7e52-48ec-9bb8-d661890a3a9f-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:33:34.276848 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.276820    2581 generic.go:356] "Generic (PLEG): container finished" podID="883854de-8b63-45da-9763-f73dea0cb887" containerID="5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb" exitCode=0
Sep 06 19:33:34.277038 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.276888    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" event={"ID":"883854de-8b63-45da-9763-f73dea0cb887","Type":"ContainerDied","Data":"5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb"}
Sep 06 19:33:34.277355 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.277328    2581 scope.go:117] "RemoveContainer" containerID="5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb"
Sep 06 19:33:34.278477 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.278450    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:34.278583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.278530    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:34.278583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.278575    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:34.278810 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.278794    2581 generic.go:356] "Generic (PLEG): container finished" podID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerID="f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec" exitCode=0
Sep 06 19:33:34.278853 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.278833    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"
Sep 06 19:33:34.278899 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.278880    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" event={"ID":"2206bafb-7e52-48ec-9bb8-d661890a3a9f","Type":"ContainerDied","Data":"f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec"}
Sep 06 19:33:34.278935 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.278928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2" event={"ID":"2206bafb-7e52-48ec-9bb8-d661890a3a9f","Type":"ContainerDied","Data":"023d7648dec17a2b4aa743230e3bdda62a54d49665a82013d3dcb9fec87ed83b"}
Sep 06 19:33:34.278970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.278952    2581 scope.go:117] "RemoveContainer" containerID="f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec"
Sep 06 19:33:34.291018 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.290996    2581 scope.go:117] "RemoveContainer" containerID="4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36"
Sep 06 19:33:34.311156 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.311122    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"]
Sep 06 19:33:34.312366 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.312313    2581 scope.go:117] "RemoveContainer" containerID="f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec"
Sep 06 19:33:34.312864 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:33:34.312829    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec\": container with ID starting with f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec not found: ID does not exist" containerID="f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec"
Sep 06 19:33:34.312971 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.312878    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec"} err="failed to get container status \"f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec\": rpc error: code = NotFound desc = could not find container \"f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec\": container with ID starting with f0eae624a51ca80d5aed53ef1146e247aa0e57e9eaa90ce9cfd3d24cebcbb0ec not found: ID does not exist"
Sep 06 19:33:34.312971 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.312907    2581 scope.go:117] "RemoveContainer" containerID="4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36"
Sep 06 19:33:34.313793 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:33:34.313763    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36\": container with ID starting with 4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36 not found: ID does not exist" containerID="4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36"
Sep 06 19:33:34.313871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.313804    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36"} err="failed to get container status \"4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36\": rpc error: code = NotFound desc = could not find container \"4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36\": container with ID starting with 4d4de64e4c2b3b5c80fbc14ae18cc700aa412fdb40d678bc66263fdf2eff6e36 not found: ID does not exist"
Sep 06 19:33:34.314672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:34.314650    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d73678-openshift-default-85c8f94f9d-l44f2"]
Sep 06 19:33:35.288435 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:35.288392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" event={"ID":"883854de-8b63-45da-9763-f73dea0cb887","Type":"ContainerStarted","Data":"c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4"}
Sep 06 19:33:35.642127 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:35.642093    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:35.642333 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:35.642312    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 19:33:35.642386 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:35.642362    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 19:33:36.266660 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:36.266623    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" path="/var/lib/kubelet/pods/2206bafb-7e52-48ec-9bb8-d661890a3a9f/volumes"
Sep 06 19:33:36.642429 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:36.642391    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 19:33:36.642805 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:36.642457    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 19:33:37.642836 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:37.642787    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 19:33:37.643304 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:37.642850    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 19:33:38.642686 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:38.642650    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 19:33:38.642960 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:38.642937    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 19:33:39.642831 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:39.642787    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 19:33:39.643049 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:39.642871    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 19:33:40.022105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:40.021976    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:40.752790 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:40.752757    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:40.753509 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:40.753435    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:40.754732 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:40.754672    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:33:41.950365 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:41.950335    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:46.825415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:46.824333    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": read tcp 10.134.0.2:54100->10.134.0.69:15021: read: connection reset by peer" start-of-body=
Sep 06 19:33:46.825415 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:46.824413    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": read tcp 10.134.0.2:54100->10.134.0.69:15021: read: connection reset by peer"
Sep 06 19:33:47.380497 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:47.380457    2581 generic.go:356] "Generic (PLEG): container finished" podID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerID="3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3" exitCode=0
Sep 06 19:33:47.380717 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:47.380533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" event={"ID":"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42","Type":"ContainerDied","Data":"3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3"}
Sep 06 19:33:47.381093 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:47.381049    2581 scope.go:117] "RemoveContainer" containerID="3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3"
Sep 06 19:33:47.382375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:47.382346    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:47.382490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:47.382440    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:47.382559 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:47.382491    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:48.390219 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:48.390180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" event={"ID":"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42","Type":"ContainerStarted","Data":"ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10"}
Sep 06 19:33:48.845490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:48.845381    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:48.845649 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:48.845627    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 06 19:33:48.845698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:48.845681    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 06 19:33:49.845466 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:49.845429    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 06 19:33:49.845850 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:49.845485    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 06 19:33:50.845504 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:50.845455    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 06 19:33:50.846001 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:50.845530    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 06 19:33:51.845943 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:51.845905    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 06 19:33:51.846422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:51.845964    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 06 19:33:52.845132 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:52.845092    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 06 19:33:52.845332 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:52.845164    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 06 19:33:53.848434 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:53.848403    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:53.848844 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:53.848761    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:53.849592 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:53.849571    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:33:53.928494 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:53.928461    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:55.718939 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.718894    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:55.977374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.977254    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"]
Sep 06 19:33:55.977833 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.977812    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy"
Sep 06 19:33:55.977958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.977835    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy"
Sep 06 19:33:55.977958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.977850    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy"
Sep 06 19:33:55.977958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.977858    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy"
Sep 06 19:33:55.978158 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.977990    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e562473-4e05-4539-8d95-b4c0a7439b13" containerName="istio-proxy"
Sep 06 19:33:55.978158 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.978007    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy"
Sep 06 19:33:55.981578 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.981557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:55.983616 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.983583    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-84ba6496-openshift-default-dockercfg-9vg9q\""
Sep 06 19:33:55.992616 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:55.992586    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"]
Sep 06 19:33:56.006234 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.006199    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:56.107476 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.107438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-credential-socket\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.107630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.107502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-istio-token\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.107630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.107561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-certs\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.107630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.107609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-486kd\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-kube-api-access-486kd\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.107758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.107647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-socket\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.107758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.107682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25fa22df-8198-42f6-9018-86f807c13a3f-istio-podinfo\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.107758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.107714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25fa22df-8198-42f6-9018-86f807c13a3f-istiod-ca-cert\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.107857 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.107776    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-envoy\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.107857 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.107799    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-data\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.108510 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.108438    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:56.209185 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-socket\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25fa22df-8198-42f6-9018-86f807c13a3f-istio-podinfo\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25fa22df-8198-42f6-9018-86f807c13a3f-istiod-ca-cert\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-envoy\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209475 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209415    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-data\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209523 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-credential-socket\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209569 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-istio-token\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209650 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-certs\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209703 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-486kd\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-kube-api-access-486kd\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209767 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209718    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-data\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.209767 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-socket\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.210041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-credential-socket\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.210041 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.209992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-certs\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.210406 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.210258    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25fa22df-8198-42f6-9018-86f807c13a3f-istiod-ca-cert\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.211786 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.211755    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-envoy\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.211980 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.211962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25fa22df-8198-42f6-9018-86f807c13a3f-istio-podinfo\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.216638 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.216612    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-istio-token\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.216874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.216856    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-486kd\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-kube-api-access-486kd\") pod \"e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") " pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.284576 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.284458    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:56.297466 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.297435    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:56.386618 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.386580    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:56.579110 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.579076    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"]
Sep 06 19:33:56.585459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.585423    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:56.585596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.585508    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:56.585596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.585551    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:56.592203 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.592172    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"]
Sep 06 19:33:56.592992 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.592924    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2206bafb-7e52-48ec-9bb8-d661890a3a9f" containerName="istio-proxy"
Sep 06 19:33:56.596245 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.596129    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.598601 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.598135    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-476f98-openshift-default-dockercfg-qngz2\""
Sep 06 19:33:56.608485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.608454    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"]
Sep 06 19:33:56.628319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.628279    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:56.718722 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.718683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-socket\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.719138 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.719046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-envoy\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.719676 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.719655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-credential-socket\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.719824 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.719808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxqx4\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-kube-api-access-dxqx4\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.719983 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.719966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-certs\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.720227 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.720208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-istio-token\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.720398 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.720372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-data\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.720532 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.720515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83122217-d60d-4ef3-9762-8d24d44641d8-istio-podinfo\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.720718 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.720684    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83122217-d60d-4ef3-9762-8d24d44641d8-istiod-ca-cert\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-istio-token\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822313    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-data\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83122217-d60d-4ef3-9762-8d24d44641d8-istio-podinfo\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822642 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83122217-d60d-4ef3-9762-8d24d44641d8-istiod-ca-cert\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822642 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-socket\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822654    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-envoy\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-data\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822869 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-credential-socket\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822927 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxqx4\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-kube-api-access-dxqx4\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.822927 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-certs\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.823042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.822993    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-socket\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.823298 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.823273    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-credential-socket\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.823417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.823336    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-certs\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.823746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.823721    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83122217-d60d-4ef3-9762-8d24d44641d8-istiod-ca-cert\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.825167 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.825148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-envoy\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.825777 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.825756    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83122217-d60d-4ef3-9762-8d24d44641d8-istio-podinfo\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.830209 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.830155    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-istio-token\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.830412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.830381    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxqx4\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-kube-api-access-dxqx4\") pod \"e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") " pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.911715 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.911673    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:56.926313 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:56.926286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:57.188740 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.188710    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"]
Sep 06 19:33:57.190806 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:33:57.190776    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83122217_d60d_4ef3_9762_8d24d44641d8.slice/crio-3d09f726f5b77c3ecdff6731a3d3eeb360947c0595372372875418433b210f85 WatchSource:0}: Error finding container 3d09f726f5b77c3ecdff6731a3d3eeb360947c0595372372875418433b210f85: Status 404 returned error can't find the container with id 3d09f726f5b77c3ecdff6731a3d3eeb360947c0595372372875418433b210f85
Sep 06 19:33:57.193247 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.193216    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:57.193317 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.193283    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:57.193317 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.193310    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:33:57.455485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.455384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" event={"ID":"83122217-d60d-4ef3-9762-8d24d44641d8","Type":"ContainerStarted","Data":"d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa"}
Sep 06 19:33:57.455485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.455446    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" event={"ID":"83122217-d60d-4ef3-9762-8d24d44641d8","Type":"ContainerStarted","Data":"3d09f726f5b77c3ecdff6731a3d3eeb360947c0595372372875418433b210f85"}
Sep 06 19:33:57.457388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.457359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" event={"ID":"25fa22df-8198-42f6-9018-86f807c13a3f","Type":"ContainerStarted","Data":"32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85"}
Sep 06 19:33:57.457557 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.457412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" event={"ID":"25fa22df-8198-42f6-9018-86f807c13a3f","Type":"ContainerStarted","Data":"e3827e14fc9a31f3827c4e9fb5a415948fe528a9bfd45e47012eb98b6635b461"}
Sep 06 19:33:57.474801 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.474748    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" podStartSLOduration=1.474731454 podStartE2EDuration="1.474731454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:33:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:33:57.472482815 +0000 UTC m=+1365.752935612" watchObservedRunningTime="2026-09-06 19:33:57.474731454 +0000 UTC m=+1365.755184328"
Sep 06 19:33:57.494357 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.494287    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" podStartSLOduration=2.494254982 podStartE2EDuration="2.494254982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:33:57.489784966 +0000 UTC m=+1365.770237757" watchObservedRunningTime="2026-09-06 19:33:57.494254982 +0000 UTC m=+1365.774707782"
Sep 06 19:33:57.911805 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:57.911772    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:58.298444 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:58.298346    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:58.303803 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:58.303777    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:58.463765 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:58.463735    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:58.464844 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:58.464823    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:33:58.917699 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:58.917666    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:59.351487 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:59.351396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:59.454001 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:59.453922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:33:59.469721 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:59.469695    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:59.470631 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:59.470610    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:33:59.549457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:33:59.549423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:00.493941 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:00.493900    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:00.594082 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:00.594036    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:00.744330 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:00.744245    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:00.966412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:00.966360    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:01.063043 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:01.062955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:01.460368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:01.460331    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:01.560222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:01.560189    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:01.661379 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:01.661345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:01.898292 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:01.898259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:02.000960 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:02.000926    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:02.280118 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:02.279987    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:03.001843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:03.001812    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:10.406758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:10.406704    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:10.506720 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:10.506689    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:12.376887 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:12.376852    2581 scope.go:117] "RemoveContainer" containerID="0c1c52204e405c5ff69cd183fd18f1a7dda880273432309bce2d93bda1e5c7ac"
Sep 06 19:34:19.623028 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.622918    2581 generic.go:356] "Generic (PLEG): container finished" podID="83122217-d60d-4ef3-9762-8d24d44641d8" containerID="d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa" exitCode=0
Sep 06 19:34:19.623028 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.622991    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" event={"ID":"83122217-d60d-4ef3-9762-8d24d44641d8","Type":"ContainerDied","Data":"d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa"}
Sep 06 19:34:19.623593 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.623488    2581 scope.go:117] "RemoveContainer" containerID="d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa"
Sep 06 19:34:19.624800 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.624766    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:34:19.624928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.624836    2581 generic.go:356] "Generic (PLEG): container finished" podID="25fa22df-8198-42f6-9018-86f807c13a3f" containerID="32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85" exitCode=0
Sep 06 19:34:19.624928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.624845    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:34:19.625046 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.624924    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" event={"ID":"25fa22df-8198-42f6-9018-86f807c13a3f","Type":"ContainerDied","Data":"32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85"}
Sep 06 19:34:19.625046 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.624941    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:34:19.625330 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.625316    2581 scope.go:117] "RemoveContainer" containerID="32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85"
Sep 06 19:34:19.626390 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.626359    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:34:19.626538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.626438    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:34:19.626538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:19.626476    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:34:20.637569 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:20.637537    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" event={"ID":"83122217-d60d-4ef3-9762-8d24d44641d8","Type":"ContainerStarted","Data":"8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9"}
Sep 06 19:34:20.640490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:20.640456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" event={"ID":"25fa22df-8198-42f6-9018-86f807c13a3f","Type":"ContainerStarted","Data":"9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b"}
Sep 06 19:34:20.912384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:20.912283    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:34:20.912742 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:20.912715    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 06 19:34:20.912918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:20.912893    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 06 19:34:21.298230 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:21.298075    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:34:21.298643 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:21.298616    2581 patch_prober.go:28] interesting pod/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 19:34:21.298844 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:21.298816    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 19:34:21.470788 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:21.470753    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:21.912289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:21.912248    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 06 19:34:21.912902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:21.912863    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 06 19:34:22.298794 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:22.298689    2581 patch_prober.go:28] interesting pod/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 19:34:22.299090 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:22.299034    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 19:34:22.434982 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:22.434926    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:22.912915 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:22.912873    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 06 19:34:22.913583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:22.913548    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 06 19:34:23.065405 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:23.065349    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:23.298829 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:23.298724    2581 patch_prober.go:28] interesting pod/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 19:34:23.298829 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:23.298794    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 19:34:23.669810 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:23.669775    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:23.913109 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:23.913042    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 06 19:34:23.913732 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:23.913682    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 06 19:34:24.282907 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:24.282855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:24.297921 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:24.297873    2581 patch_prober.go:28] interesting pod/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 19:34:24.298161 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:24.297948    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 19:34:24.384550 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:24.384493    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:24.912254 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:24.912208    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 06 19:34:24.912453 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:24.912286    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 06 19:34:25.298349 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:25.298247    2581 patch_prober.go:28] interesting pod/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 19:34:25.299342 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:25.298903    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 19:34:26.238103 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:26.238048    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:34:26.238648 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:26.238560    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:34:26.239628 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:26.239607    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:34:26.302335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:26.302303    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:34:26.302798 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:26.302783    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:34:26.303537 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:26.303518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:34:31.065127 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:31.065094    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:34:48.610456 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:34:48.610390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:03.672643 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:03.672555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:08.636046 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.636006    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"]
Sep 06 19:35:08.639956 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.639939    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"
Sep 06 19:35:08.642102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.642054    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:35:08.642817 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.642793    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:35:08.646222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.646196    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"]
Sep 06 19:35:08.710325 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.710286    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd987964-c705-437a-8088-70d8310393bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z\" (UID: \"bd987964-c705-437a-8088-70d8310393bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"
Sep 06 19:35:08.769649 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.769615    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:08.811189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.811147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd987964-c705-437a-8088-70d8310393bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z\" (UID: \"bd987964-c705-437a-8088-70d8310393bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"
Sep 06 19:35:08.811817 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.811793    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd987964-c705-437a-8088-70d8310393bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z\" (UID: \"bd987964-c705-437a-8088-70d8310393bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"
Sep 06 19:35:08.952944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:08.952853    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"
Sep 06 19:35:09.245666 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:09.245638    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"]
Sep 06 19:35:09.246975 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:35:09.246948    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd987964_c705_437a_8088_70d8310393bb.slice/crio-ccf70bee9f377062ded8eac2dab65a2265eed21b0faaa4a42df54a36168234d5 WatchSource:0}: Error finding container ccf70bee9f377062ded8eac2dab65a2265eed21b0faaa4a42df54a36168234d5: Status 404 returned error can't find the container with id ccf70bee9f377062ded8eac2dab65a2265eed21b0faaa4a42df54a36168234d5
Sep 06 19:35:09.970990 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:09.970953    2581 generic.go:356] "Generic (PLEG): container finished" podID="bd987964-c705-437a-8088-70d8310393bb" containerID="e9cb566434f4303f1ca12642c9d13f4e931c85bb344eda2b06e47f9fd21775f8" exitCode=0
Sep 06 19:35:09.971383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:09.971034    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z" event={"ID":"bd987964-c705-437a-8088-70d8310393bb","Type":"ContainerDied","Data":"e9cb566434f4303f1ca12642c9d13f4e931c85bb344eda2b06e47f9fd21775f8"}
Sep 06 19:35:09.971383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:09.971146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z" event={"ID":"bd987964-c705-437a-8088-70d8310393bb","Type":"ContainerStarted","Data":"ccf70bee9f377062ded8eac2dab65a2265eed21b0faaa4a42df54a36168234d5"}
Sep 06 19:35:11.115896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:11.115869    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"
Sep 06 19:35:11.237871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:11.237777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd987964-c705-437a-8088-70d8310393bb-maas-api-service-ca\") pod \"bd987964-c705-437a-8088-70d8310393bb\" (UID: \"bd987964-c705-437a-8088-70d8310393bb\") "
Sep 06 19:35:11.238258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:11.238228    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd987964-c705-437a-8088-70d8310393bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bd987964-c705-437a-8088-70d8310393bb" (UID: "bd987964-c705-437a-8088-70d8310393bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:35:11.339003 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:11.338959    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd987964-c705-437a-8088-70d8310393bb-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:35:11.984665 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:11.984638    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"
Sep 06 19:35:11.984838 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:11.984639    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z" event={"ID":"bd987964-c705-437a-8088-70d8310393bb","Type":"ContainerDied","Data":"ccf70bee9f377062ded8eac2dab65a2265eed21b0faaa4a42df54a36168234d5"}
Sep 06 19:35:11.984838 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:11.984763    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccf70bee9f377062ded8eac2dab65a2265eed21b0faaa4a42df54a36168234d5"
Sep 06 19:35:12.426832 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:12.426792    2581 scope.go:117] "RemoveContainer" containerID="ebcf102662f33864e451aba4b638e51a4bc113928bccc3eb2cd30aaa71afaea1"
Sep 06 19:35:23.646787 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.646734    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"]
Sep 06 19:35:23.647424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.647385    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd987964-c705-437a-8088-70d8310393bb" containerName="revoke-keys"
Sep 06 19:35:23.647424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.647403    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd987964-c705-437a-8088-70d8310393bb" containerName="revoke-keys"
Sep 06 19:35:23.647499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.647473    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd987964-c705-437a-8088-70d8310393bb" containerName="revoke-keys"
Sep 06 19:35:23.651983 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.651965    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.654257 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.654230    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-476f98\"/\"kube-root-ca.crt\""
Sep 06 19:35:23.654422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.654237    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-476f98\"/\"openshift-service-ca.crt\""
Sep 06 19:35:23.654422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.654295    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-476f98\"/\"default-dockercfg-lf6cj\""
Sep 06 19:35:23.654422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.654295    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-476f98\"/\"auth-test-model-476f98-kserve-self-signed-certs\""
Sep 06 19:35:23.659639 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.659619    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"]
Sep 06 19:35:23.744825 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.744794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:23.772024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.771989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-model-cache\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.772219 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.772109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-tmp-dir\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.772219 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.772181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbfjv\" (UniqueName: \"kubernetes.io/projected/90aaaf4a-972a-4355-875a-00284c71529d-kube-api-access-xbfjv\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.772340 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.772242    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90aaaf4a-972a-4355-875a-00284c71529d-tls-certs\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.772396 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.772349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-dshm\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.772447 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.772404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-home\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.845376 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.845319    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:23.873619 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.873580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-dshm\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.873831 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.873658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-home\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.873831 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.873701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-model-cache\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.873831 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.873742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-tmp-dir\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.873831 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.873791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbfjv\" (UniqueName: \"kubernetes.io/projected/90aaaf4a-972a-4355-875a-00284c71529d-kube-api-access-xbfjv\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.874467 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.873842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90aaaf4a-972a-4355-875a-00284c71529d-tls-certs\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.874467 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.874137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-home\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.874467 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.874172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-model-cache\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.874467 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.874300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-tmp-dir\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.876059 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.876037    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-dshm\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.876516 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.876496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90aaaf4a-972a-4355-875a-00284c71529d-tls-certs\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.881195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.881171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbfjv\" (UniqueName: \"kubernetes.io/projected/90aaaf4a-972a-4355-875a-00284c71529d-kube-api-access-xbfjv\") pod \"auth-test-model-476f98-kserve-5749c888c9-84n4j\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") " pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:23.946501 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.946392    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:23.963222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:23.963192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:24.059919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:24.059878    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:24.160958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:24.160923    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:24.232973 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:24.232828    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"]
Sep 06 19:35:24.235102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:24.234977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:24.235986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:24.235964    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-84ba6496-kzg5z"]
Sep 06 19:35:24.252815 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:24.252791    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"]
Sep 06 19:35:24.254592 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:35:24.254558    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90aaaf4a_972a_4355_875a_00284c71529d.slice/crio-a006ef03a05c28dc250107f11d9603e3aff602e883ef477f1e6413025879318e WatchSource:0}: Error finding container a006ef03a05c28dc250107f11d9603e3aff602e883ef477f1e6413025879318e: Status 404 returned error can't find the container with id a006ef03a05c28dc250107f11d9603e3aff602e883ef477f1e6413025879318e
Sep 06 19:35:24.268075 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:24.268037    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd987964-c705-437a-8088-70d8310393bb" path="/var/lib/kubelet/pods/bd987964-c705-437a-8088-70d8310393bb/volumes"
Sep 06 19:35:24.291722 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:24.291664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:25.069762 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:25.069725    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j" event={"ID":"90aaaf4a-972a-4355-875a-00284c71529d","Type":"ContainerStarted","Data":"29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6"}
Sep 06 19:35:25.070203 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:25.069770    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j" event={"ID":"90aaaf4a-972a-4355-875a-00284c71529d","Type":"ContainerStarted","Data":"a006ef03a05c28dc250107f11d9603e3aff602e883ef477f1e6413025879318e"}
Sep 06 19:35:25.086960 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:25.086892    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j" podStartSLOduration=2.086868625 podStartE2EDuration="2.086868625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:35:25.084610486 +0000 UTC m=+1453.365063281" watchObservedRunningTime="2026-09-06 19:35:25.086868625 +0000 UTC m=+1453.367321420"
Sep 06 19:35:27.239546 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:27.239506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:32.264263 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:32.264232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:35:33.963656 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:33.963592    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:33.964112 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:33.963679    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:33.976653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:33.976625    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:35:34.135981 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:35:34.135953    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:36:08.577943 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:08.577905    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:09.294294 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:09.294263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:12.349561 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:12.349528    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:36:12.355437 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:12.355411    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:36:12.355592 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:12.355489    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:36:12.364953 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:12.364930    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:36:12.450607 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:12.450577    2581 scope.go:117] "RemoveContainer" containerID="3f205ed779876216870243b79de138e626cfd9c960d1a789385dbc4d6c98b6c3"
Sep 06 19:36:12.460292 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:12.460268    2581 scope.go:117] "RemoveContainer" containerID="e404e7f0401020f8a7da926cfa358aa55875ae38b4a8b27a59868d7b10ff96c1"
Sep 06 19:36:12.469220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:12.469199    2581 scope.go:117] "RemoveContainer" containerID="73ca7b1c29c0538e6ca5d758cd0a2b42cc3b20be406fb84c1b6a8c10eb0b9a43"
Sep 06 19:36:12.478213 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:12.478192    2581 scope.go:117] "RemoveContainer" containerID="645de9b37df7850c9761eaf81eb1be6987b0d88b5f3fb379cc6260651453e32f"
Sep 06 19:36:14.837221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:14.837182    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:15.058920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:15.058874    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"]
Sep 06 19:36:15.059214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:15.059179    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy" containerID="cri-o://9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b" gracePeriod=30
Sep 06 19:36:16.026850 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:16.026816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:16.256494 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:16.256462    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:16.358475 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:16.358386    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:16.455795 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:16.455760    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:17.029281 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:17.029237    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:17.126251 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:17.126216    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:17.226009 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:17.225975    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:18.494967 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:18.494936    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:20.322181 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.322120    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:36:20.375876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.375840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-socket\") pod \"25fa22df-8198-42f6-9018-86f807c13a3f\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") "
Sep 06 19:36:20.376054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.375920    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-certs\") pod \"25fa22df-8198-42f6-9018-86f807c13a3f\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") "
Sep 06 19:36:20.376054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.375950    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25fa22df-8198-42f6-9018-86f807c13a3f-istio-podinfo\") pod \"25fa22df-8198-42f6-9018-86f807c13a3f\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") "
Sep 06 19:36:20.376054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376003    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25fa22df-8198-42f6-9018-86f807c13a3f-istiod-ca-cert\") pod \"25fa22df-8198-42f6-9018-86f807c13a3f\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") "
Sep 06 19:36:20.376054 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376030    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-data\") pod \"25fa22df-8198-42f6-9018-86f807c13a3f\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") "
Sep 06 19:36:20.376306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376116    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-486kd\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-kube-api-access-486kd\") pod \"25fa22df-8198-42f6-9018-86f807c13a3f\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") "
Sep 06 19:36:20.376306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376132    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "25fa22df-8198-42f6-9018-86f807c13a3f" (UID: "25fa22df-8198-42f6-9018-86f807c13a3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:36:20.376306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376184    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-envoy\") pod \"25fa22df-8198-42f6-9018-86f807c13a3f\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") "
Sep 06 19:36:20.376306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376222    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-istio-token\") pod \"25fa22df-8198-42f6-9018-86f807c13a3f\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") "
Sep 06 19:36:20.376306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376265    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-credential-socket\") pod \"25fa22df-8198-42f6-9018-86f807c13a3f\" (UID: \"25fa22df-8198-42f6-9018-86f807c13a3f\") "
Sep 06 19:36:20.376306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376283    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "25fa22df-8198-42f6-9018-86f807c13a3f" (UID: "25fa22df-8198-42f6-9018-86f807c13a3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:36:20.376622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376362    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-data" (OuterVolumeSpecName: "istio-data") pod "25fa22df-8198-42f6-9018-86f807c13a3f" (UID: "25fa22df-8198-42f6-9018-86f807c13a3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:36:20.376622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376563    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "25fa22df-8198-42f6-9018-86f807c13a3f" (UID: "25fa22df-8198-42f6-9018-86f807c13a3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:36:20.376902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376684    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:36:20.376902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376686    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25fa22df-8198-42f6-9018-86f807c13a3f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "25fa22df-8198-42f6-9018-86f807c13a3f" (UID: "25fa22df-8198-42f6-9018-86f807c13a3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:36:20.376902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376697    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:36:20.376902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376735    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:36:20.376902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.376754    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:36:20.378490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.378458    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25fa22df-8198-42f6-9018-86f807c13a3f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "25fa22df-8198-42f6-9018-86f807c13a3f" (UID: "25fa22df-8198-42f6-9018-86f807c13a3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:36:20.378613 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.378555    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-kube-api-access-486kd" (OuterVolumeSpecName: "kube-api-access-486kd") pod "25fa22df-8198-42f6-9018-86f807c13a3f" (UID: "25fa22df-8198-42f6-9018-86f807c13a3f"). InnerVolumeSpecName "kube-api-access-486kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:36:20.378737 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.378720    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-istio-token" (OuterVolumeSpecName: "istio-token") pod "25fa22df-8198-42f6-9018-86f807c13a3f" (UID: "25fa22df-8198-42f6-9018-86f807c13a3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:36:20.379049 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.379029    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "25fa22df-8198-42f6-9018-86f807c13a3f" (UID: "25fa22df-8198-42f6-9018-86f807c13a3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:36:20.418167 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.418126    2581 generic.go:356] "Generic (PLEG): container finished" podID="25fa22df-8198-42f6-9018-86f807c13a3f" containerID="9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b" exitCode=0
Sep 06 19:36:20.418361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.418198    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"
Sep 06 19:36:20.418361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.418209    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" event={"ID":"25fa22df-8198-42f6-9018-86f807c13a3f","Type":"ContainerDied","Data":"9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b"}
Sep 06 19:36:20.418361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.418301    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk" event={"ID":"25fa22df-8198-42f6-9018-86f807c13a3f","Type":"ContainerDied","Data":"e3827e14fc9a31f3827c4e9fb5a415948fe528a9bfd45e47012eb98b6635b461"}
Sep 06 19:36:20.418361 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.418325    2581 scope.go:117] "RemoveContainer" containerID="9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b"
Sep 06 19:36:20.429497 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.429478    2581 scope.go:117] "RemoveContainer" containerID="32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85"
Sep 06 19:36:20.439247 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.439226    2581 scope.go:117] "RemoveContainer" containerID="9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b"
Sep 06 19:36:20.439609 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:36:20.439583    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b\": container with ID starting with 9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b not found: ID does not exist" containerID="9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b"
Sep 06 19:36:20.439675 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.439622    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b"} err="failed to get container status \"9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b\": rpc error: code = NotFound desc = could not find container \"9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b\": container with ID starting with 9d9f7458926786c792fa455a66a7dcb63d924bcee48fe837e9721d7aa663356b not found: ID does not exist"
Sep 06 19:36:20.439675 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.439648    2581 scope.go:117] "RemoveContainer" containerID="32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85"
Sep 06 19:36:20.439971 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:36:20.439954    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85\": container with ID starting with 32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85 not found: ID does not exist" containerID="32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85"
Sep 06 19:36:20.440013 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.439977    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85"} err="failed to get container status \"32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85\": rpc error: code = NotFound desc = could not find container \"32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85\": container with ID starting with 32855ac74d7a97105a773f84eb16be76623ad4ac45c8bbf9b7edacf6e07d0c85 not found: ID does not exist"
Sep 06 19:36:20.440426 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.440410    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"]
Sep 06 19:36:20.443569 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.443545    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-84ba6496-openshift-default-8b9c7b84-lvvmk"]
Sep 06 19:36:20.477547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.477512    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25fa22df-8198-42f6-9018-86f807c13a3f-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:36:20.477547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.477541    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25fa22df-8198-42f6-9018-86f807c13a3f-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:36:20.477547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.477552    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-486kd\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-kube-api-access-486kd\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:36:20.477754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.477562    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25fa22df-8198-42f6-9018-86f807c13a3f-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:36:20.477754 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:20.477571    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25fa22df-8198-42f6-9018-86f807c13a3f-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:36:22.266190 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:22.266152    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" path="/var/lib/kubelet/pods/25fa22df-8198-42f6-9018-86f807c13a3f/volumes"
Sep 06 19:36:24.157330 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:24.157294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:24.257476 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:24.257444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:24.804516 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:24.804481    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:30.514391 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:30.514356    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:30.570606 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:30.570575    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:30.652322 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:30.652287    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:30.746610 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:30.746577    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:41.189227 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:41.189185    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:41.404958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:41.404920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:44.006476 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:44.006441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:36:52.088462 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:36:52.088430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:02.280344 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:02.280299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:07.841375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:07.841340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:07.903279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:07.903244    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:08.158329 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.158294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:08.431503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.431421    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"]
Sep 06 19:37:08.431950 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.431932    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy"
Sep 06 19:37:08.432044 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.431953    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy"
Sep 06 19:37:08.432044 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.431966    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy"
Sep 06 19:37:08.432044 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.431973    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy"
Sep 06 19:37:08.432220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.432114    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy"
Sep 06 19:37:08.432220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.432134    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="25fa22df-8198-42f6-9018-86f807c13a3f" containerName="istio-proxy"
Sep 06 19:37:08.435643 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.435621    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.437883 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.437863    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c31435-openshift-default-dockercfg-jw62b\""
Sep 06 19:37:08.447814 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.447786    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"]
Sep 06 19:37:08.451903 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.451879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:08.578617 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.578582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-podinfo\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.578617 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.578626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6lnd\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-kube-api-access-r6lnd\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.578855 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.578699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-data\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.578855 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.578736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcdcde74-66c4-4222-aca0-69db39e9163b-istiod-ca-cert\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.578855 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.578758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-credential-socket\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.578855 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.578846    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-token\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.579007 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.578882    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-certs\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.579007 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.578914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-envoy\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.579007 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.578977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-socket\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680341 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-podinfo\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680341 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6lnd\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-kube-api-access-r6lnd\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-data\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcdcde74-66c4-4222-aca0-69db39e9163b-istiod-ca-cert\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-credential-socket\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-token\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-certs\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680595    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-envoy\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680936 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-socket\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.680936 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680845    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-data\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.681052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-credential-socket\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.681052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.680974    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-certs\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.681052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.681023    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-socket\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.681388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.681372    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcdcde74-66c4-4222-aca0-69db39e9163b-istiod-ca-cert\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.683013 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.682956    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-podinfo\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.683013 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.682973    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-envoy\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.687777 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.687757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6lnd\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-kube-api-access-r6lnd\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.687879 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.687777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-token\") pod \"e2e-api-a-c31435-openshift-default-668ff779f8-gszhv\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") " pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.748228 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.748190    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:08.828961 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:08.828929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:09.023742 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:09.023713    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"]
Sep 06 19:37:09.025766 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:37:09.025739    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcdcde74_66c4_4222_aca0_69db39e9163b.slice/crio-2a9adbfb3e19ef036859726e6aac35dfa46038a78aed240f7d1546344e708313 WatchSource:0}: Error finding container 2a9adbfb3e19ef036859726e6aac35dfa46038a78aed240f7d1546344e708313: Status 404 returned error can't find the container with id 2a9adbfb3e19ef036859726e6aac35dfa46038a78aed240f7d1546344e708313
Sep 06 19:37:09.028596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:09.028559    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:09.028667 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:09.028634    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:09.028710 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:09.028665    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:09.706256 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:09.706217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" event={"ID":"bcdcde74-66c4-4222-aca0-69db39e9163b","Type":"ContainerStarted","Data":"5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef"}
Sep 06 19:37:09.706433 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:09.706281    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" event={"ID":"bcdcde74-66c4-4222-aca0-69db39e9163b","Type":"ContainerStarted","Data":"2a9adbfb3e19ef036859726e6aac35dfa46038a78aed240f7d1546344e708313"}
Sep 06 19:37:09.725533 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:09.725479    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" podStartSLOduration=1.725461652 podStartE2EDuration="1.725461652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:37:09.72348651 +0000 UTC m=+1558.003939303" watchObservedRunningTime="2026-09-06 19:37:09.725461652 +0000 UTC m=+1558.005914445"
Sep 06 19:37:09.748403 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:09.748367    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:10.754737 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:10.754697    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:11.593333 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:11.593286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:11.660366 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:11.660330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:11.721835 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:11.721804    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:11.722713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:11.722692    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:12.513916 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:12.513881    2581 scope.go:117] "RemoveContainer" containerID="c3aa14dbd4625aaa00638a8d59f865b4c6fc2af2db64dabaf13ed3d3822f2cbc"
Sep 06 19:37:12.780758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:12.780680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:12.877327 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:12.877290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:13.179459 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:13.179415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:13.279076 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:13.279030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:13.359481 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:13.359439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:18.706522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:18.706491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:19.524541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:19.524509    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"]
Sep 06 19:37:19.551881 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:19.551840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:31.853647 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:31.853535    2581 generic.go:356] "Generic (PLEG): container finished" podID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerID="5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef" exitCode=0
Sep 06 19:37:31.853647 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:31.853612    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" event={"ID":"bcdcde74-66c4-4222-aca0-69db39e9163b","Type":"ContainerDied","Data":"5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef"}
Sep 06 19:37:31.854176 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:31.854087    2581 scope.go:117] "RemoveContainer" containerID="5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef"
Sep 06 19:37:31.855368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:31.855338    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:31.855458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:31.855410    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:31.855458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:31.855444    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:31.973864 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:31.973741    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:32.861862 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:32.861824    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" event={"ID":"bcdcde74-66c4-4222-aca0-69db39e9163b","Type":"ContainerStarted","Data":"22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7"}
Sep 06 19:37:33.748657 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:33.748612    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:33.748878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:33.748852    2581 patch_prober.go:28] interesting pod/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 06 19:37:33.748935 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:33.748910    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 06 19:37:34.551385 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.551345    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j" podUID="90aaaf4a-972a-4355-875a-00284c71529d" containerName="main" containerID="cri-o://29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6" gracePeriod=45
Sep 06 19:37:34.749112 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.748773    2581 patch_prober.go:28] interesting pod/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 06 19:37:34.749112 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.748852    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 06 19:37:34.843767 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.843745    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:37:34.877288 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.877257    2581 generic.go:356] "Generic (PLEG): container finished" podID="90aaaf4a-972a-4355-875a-00284c71529d" containerID="29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6" exitCode=0
Sep 06 19:37:34.877483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.877301    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j" event={"ID":"90aaaf4a-972a-4355-875a-00284c71529d","Type":"ContainerDied","Data":"29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6"}
Sep 06 19:37:34.877483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.877304    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"
Sep 06 19:37:34.877483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.877340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j" event={"ID":"90aaaf4a-972a-4355-875a-00284c71529d","Type":"ContainerDied","Data":"a006ef03a05c28dc250107f11d9603e3aff602e883ef477f1e6413025879318e"}
Sep 06 19:37:34.877483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.877361    2581 scope.go:117] "RemoveContainer" containerID="29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6"
Sep 06 19:37:34.904367 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.904246    2581 scope.go:117] "RemoveContainer" containerID="29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6"
Sep 06 19:37:34.904978 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:37:34.904899    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6\": container with ID starting with 29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6 not found: ID does not exist" containerID="29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6"
Sep 06 19:37:34.904978 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.904937    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6"} err="failed to get container status \"29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6\": rpc error: code = NotFound desc = could not find container \"29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6\": container with ID starting with 29de326d1cf2bda5e0ffda751cac10a3df2ba71bc9520d7c3eae37ab828bbfe6 not found: ID does not exist"
Sep 06 19:37:34.980670 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.980643    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-tmp-dir\") pod \"90aaaf4a-972a-4355-875a-00284c71529d\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") "
Sep 06 19:37:34.980866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.980701    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90aaaf4a-972a-4355-875a-00284c71529d-tls-certs\") pod \"90aaaf4a-972a-4355-875a-00284c71529d\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") "
Sep 06 19:37:34.980866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.980777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-model-cache\") pod \"90aaaf4a-972a-4355-875a-00284c71529d\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") "
Sep 06 19:37:34.980866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.980827    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbfjv\" (UniqueName: \"kubernetes.io/projected/90aaaf4a-972a-4355-875a-00284c71529d-kube-api-access-xbfjv\") pod \"90aaaf4a-972a-4355-875a-00284c71529d\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") "
Sep 06 19:37:34.980866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.980866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-dshm\") pod \"90aaaf4a-972a-4355-875a-00284c71529d\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") "
Sep 06 19:37:34.981111 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.980915    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-home\") pod \"90aaaf4a-972a-4355-875a-00284c71529d\" (UID: \"90aaaf4a-972a-4355-875a-00284c71529d\") "
Sep 06 19:37:34.981111 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.980929    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "90aaaf4a-972a-4355-875a-00284c71529d" (UID: "90aaaf4a-972a-4355-875a-00284c71529d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:37:34.981198 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.981112    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-model-cache" (OuterVolumeSpecName: "model-cache") pod "90aaaf4a-972a-4355-875a-00284c71529d" (UID: "90aaaf4a-972a-4355-875a-00284c71529d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:37:34.981293 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.981271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-home" (OuterVolumeSpecName: "home") pod "90aaaf4a-972a-4355-875a-00284c71529d" (UID: "90aaaf4a-972a-4355-875a-00284c71529d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:37:34.981369 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.981327    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-model-cache\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:37:34.981369 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.981341    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-tmp-dir\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:37:34.983324 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.983303    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-dshm" (OuterVolumeSpecName: "dshm") pod "90aaaf4a-972a-4355-875a-00284c71529d" (UID: "90aaaf4a-972a-4355-875a-00284c71529d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:37:34.983535 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.983515    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90aaaf4a-972a-4355-875a-00284c71529d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "90aaaf4a-972a-4355-875a-00284c71529d" (UID: "90aaaf4a-972a-4355-875a-00284c71529d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:37:34.983604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:34.983518    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90aaaf4a-972a-4355-875a-00284c71529d-kube-api-access-xbfjv" (OuterVolumeSpecName: "kube-api-access-xbfjv") pod "90aaaf4a-972a-4355-875a-00284c71529d" (UID: "90aaaf4a-972a-4355-875a-00284c71529d"). InnerVolumeSpecName "kube-api-access-xbfjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:37:35.082146 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:35.082019    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90aaaf4a-972a-4355-875a-00284c71529d-tls-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:37:35.082146 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:35.082052    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbfjv\" (UniqueName: \"kubernetes.io/projected/90aaaf4a-972a-4355-875a-00284c71529d-kube-api-access-xbfjv\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:37:35.082146 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:35.082102    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-dshm\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:37:35.082146 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:35.082116    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90aaaf4a-972a-4355-875a-00284c71529d-home\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:37:35.205216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:35.205175    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"]
Sep 06 19:37:35.210139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:35.210105    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/auth-test-model-476f98-kserve-5749c888c9-84n4j"]
Sep 06 19:37:35.748813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:35.748772    2581 patch_prober.go:28] interesting pod/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 06 19:37:35.749248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:35.748831    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 06 19:37:36.266809 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:36.266770    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90aaaf4a-972a-4355-875a-00284c71529d" path="/var/lib/kubelet/pods/90aaaf4a-972a-4355-875a-00284c71529d/volumes"
Sep 06 19:37:36.527770 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:36.527687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:36.748990 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:36.748949    2581 patch_prober.go:28] interesting pod/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 06 19:37:36.749495 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:36.749026    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 06 19:37:36.988262 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:36.988232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:37.254050 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.253952    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"]
Sep 06 19:37:37.255284 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.255259    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90aaaf4a-972a-4355-875a-00284c71529d" containerName="main"
Sep 06 19:37:37.255484 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.255458    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="90aaaf4a-972a-4355-875a-00284c71529d" containerName="main"
Sep 06 19:37:37.255871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.255852    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="90aaaf4a-972a-4355-875a-00284c71529d" containerName="main"
Sep 06 19:37:37.261714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.261688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.262416 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.262387    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"]
Sep 06 19:37:37.264183 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.264162    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-49a2d2-openshift-default-dockercfg-s5ml4\""
Sep 06 19:37:37.276417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.276395    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:37.412210 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.412158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-envoy\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.412407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.412243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rctj\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-kube-api-access-5rctj\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.412407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.412282    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-data\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.412407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.412325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-certs\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.412630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.412469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-token\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.412630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.412515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-credential-socket\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.412630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.412593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-socket\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.412630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.412626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-podinfo\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.412850 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.412654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bdb541c-74c1-484f-b7ba-449f05173a9d-istiod-ca-cert\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514188 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-socket\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514188 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-podinfo\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514188 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bdb541c-74c1-484f-b7ba-449f05173a9d-istiod-ca-cert\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514506 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-envoy\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514506 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rctj\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-kube-api-access-5rctj\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514506 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-data\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514506 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-certs\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514506 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514465    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-token\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-credential-socket\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514555    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-socket\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514669    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-data\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514775 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514733    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-certs\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514969 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-credential-socket\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.514969 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.514942    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bdb541c-74c1-484f-b7ba-449f05173a9d-istiod-ca-cert\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.516258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.516231    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:37.516866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.516849    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-envoy\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.517199 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.517181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-podinfo\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.529086 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.529051    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-token\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.529282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.529255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rctj\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-kube-api-access-5rctj\") pod \"e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") " pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.575030 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.574991    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:37.749052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.749009    2581 patch_prober.go:28] interesting pod/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 06 19:37:37.749472 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.749103    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 06 19:37:37.874226 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.874134    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"]
Sep 06 19:37:37.877113 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:37:37.877085    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bdb541c_74c1_484f_b7ba_449f05173a9d.slice/crio-a2abfd0fe520c4d94aa255eb2875dc6863a241a14db822ffa82fdcac5d6bdf71 WatchSource:0}: Error finding container a2abfd0fe520c4d94aa255eb2875dc6863a241a14db822ffa82fdcac5d6bdf71: Status 404 returned error can't find the container with id a2abfd0fe520c4d94aa255eb2875dc6863a241a14db822ffa82fdcac5d6bdf71
Sep 06 19:37:37.879513 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.879483    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:37.879598 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.879548    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:37.879598 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.879575    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:37.905915 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:37.905857    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" event={"ID":"6bdb541c-74c1-484f-b7ba-449f05173a9d","Type":"ContainerStarted","Data":"a2abfd0fe520c4d94aa255eb2875dc6863a241a14db822ffa82fdcac5d6bdf71"}
Sep 06 19:37:38.752851 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:38.752819    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:38.753344 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:38.753164    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:38.753970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:38.753950    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:37:38.913854 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:38.913819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" event={"ID":"6bdb541c-74c1-484f-b7ba-449f05173a9d","Type":"ContainerStarted","Data":"a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911"}
Sep 06 19:37:38.931809 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:38.931760    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" podStartSLOduration=1.931744243 podStartE2EDuration="1.931744243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:37:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:37:38.929600954 +0000 UTC m=+1587.210053746" watchObservedRunningTime="2026-09-06 19:37:38.931744243 +0000 UTC m=+1587.212197036"
Sep 06 19:37:39.576155 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:39.576107    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:39.581479 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:39.581452    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:39.920241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:39.920203    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:39.921097 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:39.921077    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:40.793006 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:40.792972    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:40.887940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:40.887907    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:41.931222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:41.931192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:42.038852 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:42.038810    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:42.157668 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:42.157632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:42.430955 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:42.430921    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:42.525089 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:42.525025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:50.296709 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:50.296561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:50.352393 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:50.352338    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:37:55.961362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:55.961309    2581 patch_prober.go:28] interesting pod/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 19:37:55.961790 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:55.961390    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 19:37:58.031672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:58.031637    2581 generic.go:356] "Generic (PLEG): container finished" podID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerID="a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911" exitCode=0
Sep 06 19:37:58.032115 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:58.031685    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" event={"ID":"6bdb541c-74c1-484f-b7ba-449f05173a9d","Type":"ContainerDied","Data":"a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911"}
Sep 06 19:37:58.032174 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:58.032162    2581 scope.go:117] "RemoveContainer" containerID="a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911"
Sep 06 19:37:58.033395 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:58.033369    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:58.033463 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:58.033438    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:58.033501 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:58.033469    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:37:59.039636 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:59.039598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" event={"ID":"6bdb541c-74c1-484f-b7ba-449f05173a9d","Type":"ContainerStarted","Data":"dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7"}
Sep 06 19:37:59.575945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:59.575910    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:37:59.576189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:59.576164    2581 patch_prober.go:28] interesting pod/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 06 19:37:59.576565 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:37:59.576220    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 06 19:38:00.575940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:00.575903    2581 patch_prober.go:28] interesting pod/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 06 19:38:00.576345 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:00.575959    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 06 19:38:01.575761 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:01.575723    2581 patch_prober.go:28] interesting pod/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 06 19:38:01.575934 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:01.575784    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 06 19:38:02.576317 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:02.576281    2581 patch_prober.go:28] interesting pod/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 06 19:38:02.576713 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:02.576342    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 06 19:38:03.560869 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.560782    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"]
Sep 06 19:38:03.564745 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.564720    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.567634 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.567608    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-476f98\"/\"openshift-service-ca.crt\""
Sep 06 19:38:03.567634 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.567608    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-476f98\"/\"kube-root-ca.crt\""
Sep 06 19:38:03.567845 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.567698    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-476f98\"/\"sub-test-model-476f98-kserve-self-signed-certs\""
Sep 06 19:38:03.567845 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.567789    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-476f98\"/\"default-dockercfg-lf6cj\""
Sep 06 19:38:03.574278 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.574248    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"]
Sep 06 19:38:03.576142 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.576114    2581 patch_prober.go:28] interesting pod/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 06 19:38:03.576347 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.576324    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 06 19:38:03.658275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.658240    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:03.724522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.724477    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tmp-dir\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.724522 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.724519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-home\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.724738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.724566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-model-cache\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.724738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.724629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-dshm\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.724738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.724708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tls-certs\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.724842 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.724749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp8qz\" (UniqueName: \"kubernetes.io/projected/384b1dfe-b680-4ace-a2db-b3b48bbe1006-kube-api-access-bp8qz\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.825746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.825646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-model-cache\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.825746 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.825704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-dshm\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.825967 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.825765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tls-certs\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.825967 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.825812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp8qz\" (UniqueName: \"kubernetes.io/projected/384b1dfe-b680-4ace-a2db-b3b48bbe1006-kube-api-access-bp8qz\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.825967 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.825896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tmp-dir\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.825967 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.825925    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-home\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.826279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.826252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-home\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.826349 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.826292    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tmp-dir\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.826417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.826366    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-model-cache\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.828326 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.828300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-dshm\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.828489 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.828472    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tls-certs\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.833141 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.833116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp8qz\" (UniqueName: \"kubernetes.io/projected/384b1dfe-b680-4ace-a2db-b3b48bbe1006-kube-api-access-bp8qz\") pod \"sub-test-model-476f98-kserve-6c9d97db99-tcn56\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") " pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:03.877051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:03.877010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:04.170084 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:04.166264    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"]
Sep 06 19:38:04.579499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:04.579412    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:38:04.579959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:04.579824    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:38:04.580635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:04.580619    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:38:05.077347 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:05.077313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56" event={"ID":"384b1dfe-b680-4ace-a2db-b3b48bbe1006","Type":"ContainerStarted","Data":"9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297"}
Sep 06 19:38:05.077538 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:05.077357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56" event={"ID":"384b1dfe-b680-4ace-a2db-b3b48bbe1006","Type":"ContainerStarted","Data":"2152b64edccb81d1a8e65aacb0b3bae35377c1729489d4bb522ba014b8a7f67e"}
Sep 06 19:38:05.094796 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:05.094732    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56" podStartSLOduration=2.094715426 podStartE2EDuration="2.094715426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:38:05.091758953 +0000 UTC m=+1613.372211746" watchObservedRunningTime="2026-09-06 19:38:05.094715426 +0000 UTC m=+1613.375168220"
Sep 06 19:38:05.740299 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:05.740268    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:08.506639 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.506600    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"]
Sep 06 19:38:08.510505 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.510486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"
Sep 06 19:38:08.512367 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.512347    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:38:08.512457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.512351    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:38:08.516403 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.516379    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"]
Sep 06 19:38:08.578377 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.578343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c31435-86qj5\" (UID: \"cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"
Sep 06 19:38:08.599823 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.599769    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:08.679691 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.679656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c31435-86qj5\" (UID: \"cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"
Sep 06 19:38:08.680363 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.680344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c31435-86qj5\" (UID: \"cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"
Sep 06 19:38:08.823590 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:08.823508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"
Sep 06 19:38:09.098591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:09.098562    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"]
Sep 06 19:38:09.100955 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:38:09.100926    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd13f4a7_bf04_4fd2_aac4_2088a98ba8e6.slice/crio-51c6cb219df493401f27491139fe40a008fe42ea970ed70e33313a68e7a274d7 WatchSource:0}: Error finding container 51c6cb219df493401f27491139fe40a008fe42ea970ed70e33313a68e7a274d7: Status 404 returned error can't find the container with id 51c6cb219df493401f27491139fe40a008fe42ea970ed70e33313a68e7a274d7
Sep 06 19:38:10.108791 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:10.108696    2581 generic.go:356] "Generic (PLEG): container finished" podID="cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6" containerID="e134ffc7f9dffb2c04728ad0467fc5a27c04b66071e67d6dd110e622b8267ae6" exitCode=0
Sep 06 19:38:10.109166 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:10.108780    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5" event={"ID":"cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6","Type":"ContainerDied","Data":"e134ffc7f9dffb2c04728ad0467fc5a27c04b66071e67d6dd110e622b8267ae6"}
Sep 06 19:38:10.109166 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:10.108833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5" event={"ID":"cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6","Type":"ContainerStarted","Data":"51c6cb219df493401f27491139fe40a008fe42ea970ed70e33313a68e7a274d7"}
Sep 06 19:38:11.266830 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:11.266802    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"
Sep 06 19:38:11.314811 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:11.314780    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6-maas-api-service-ca\") pod \"cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6\" (UID: \"cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6\") "
Sep 06 19:38:11.315172 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:11.315147    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6" (UID: "cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:38:11.416244 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:11.416206    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:12.122028 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:12.121995    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"
Sep 06 19:38:12.122028 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:12.122022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5" event={"ID":"cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6","Type":"ContainerDied","Data":"51c6cb219df493401f27491139fe40a008fe42ea970ed70e33313a68e7a274d7"}
Sep 06 19:38:12.122301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:12.122101    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51c6cb219df493401f27491139fe40a008fe42ea970ed70e33313a68e7a274d7"
Sep 06 19:38:12.377140 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:12.377013    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:12.552691 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:12.552657    2581 scope.go:117] "RemoveContainer" containerID="eabf539bb7259f9cd6ceea9ddd356a27035e350b63635ea32ebfec6847cd336f"
Sep 06 19:38:13.877301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:13.877251    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:13.877709 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:13.877316    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:13.890275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:13.890242    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:14.146997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:14.146961    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:38:14.256473 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:14.256433    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:18.777305 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:18.777271    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:24.311164 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:24.311123    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:24.370805 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:24.370774    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:24.470075 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:24.470041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:24.474788 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:24.474763    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"]
Sep 06 19:38:24.477426 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:24.477394    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c31435-86qj5"]
Sep 06 19:38:24.525636 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:24.525606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:24.815005 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:24.814972    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:26.266833 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:26.266790    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6" path="/var/lib/kubelet/pods/cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6/volumes"
Sep 06 19:38:30.569905 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:30.569867    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:30.629811 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:30.629770    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:30.796689 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:30.796652    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"]
Sep 06 19:38:30.797023 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:30.796991    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy" containerID="cri-o://22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7" gracePeriod=30
Sep 06 19:38:30.808967 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:30.808939    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:30.919166 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:30.919123    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:31.130050 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.130012    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"]
Sep 06 19:38:31.130549 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.130533    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6" containerName="revoke-keys"
Sep 06 19:38:31.130600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.130552    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6" containerName="revoke-keys"
Sep 06 19:38:31.130635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.130631    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd13f4a7-bf04-4fd2-aac4-2088a98ba8e6" containerName="revoke-keys"
Sep 06 19:38:31.135139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.135122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"
Sep 06 19:38:31.137696 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.137463    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:38:31.137834 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.137792    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:38:31.140524 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.140498    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"]
Sep 06 19:38:31.246837 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.246733    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1729b14-3267-442f-88e0-fd910b34dcbc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s\" (UID: \"f1729b14-3267-442f-88e0-fd910b34dcbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"
Sep 06 19:38:31.347744 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.347705    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1729b14-3267-442f-88e0-fd910b34dcbc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s\" (UID: \"f1729b14-3267-442f-88e0-fd910b34dcbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"
Sep 06 19:38:31.348407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.348390    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1729b14-3267-442f-88e0-fd910b34dcbc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s\" (UID: \"f1729b14-3267-442f-88e0-fd910b34dcbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"
Sep 06 19:38:31.384611 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.384575    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:31.449015 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.448973    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"
Sep 06 19:38:31.729692 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:31.729664    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"]
Sep 06 19:38:31.731300 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:38:31.731272    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1729b14_3267_442f_88e0_fd910b34dcbc.slice/crio-71dba3b60d12fa73a33e4d14e302cf9282aaffad6b4628ed8ecc83e0a2047cab WatchSource:0}: Error finding container 71dba3b60d12fa73a33e4d14e302cf9282aaffad6b4628ed8ecc83e0a2047cab: Status 404 returned error can't find the container with id 71dba3b60d12fa73a33e4d14e302cf9282aaffad6b4628ed8ecc83e0a2047cab
Sep 06 19:38:32.247881 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:32.247849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s" event={"ID":"f1729b14-3267-442f-88e0-fd910b34dcbc","Type":"ContainerStarted","Data":"ec9aab454c62e94e762292ec83e710e5f59a0be8a168eb3241fe03f781c6bb45"}
Sep 06 19:38:32.248035 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:32.247928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s" event={"ID":"f1729b14-3267-442f-88e0-fd910b34dcbc","Type":"ContainerStarted","Data":"71dba3b60d12fa73a33e4d14e302cf9282aaffad6b4628ed8ecc83e0a2047cab"}
Sep 06 19:38:32.262902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:32.262843    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s" podStartSLOduration=1.262827201 podStartE2EDuration="1.262827201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:38:32.260267984 +0000 UTC m=+1640.540720780" watchObservedRunningTime="2026-09-06 19:38:32.262827201 +0000 UTC m=+1640.543279995"
Sep 06 19:38:33.255544 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:33.255499    2581 generic.go:356] "Generic (PLEG): container finished" podID="f1729b14-3267-442f-88e0-fd910b34dcbc" containerID="ec9aab454c62e94e762292ec83e710e5f59a0be8a168eb3241fe03f781c6bb45" exitCode=0
Sep 06 19:38:33.255544 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:33.255549    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s" event={"ID":"f1729b14-3267-442f-88e0-fd910b34dcbc","Type":"ContainerDied","Data":"ec9aab454c62e94e762292ec83e710e5f59a0be8a168eb3241fe03f781c6bb45"}
Sep 06 19:38:34.408178 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:34.408152    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"
Sep 06 19:38:34.584451 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:34.584368    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1729b14-3267-442f-88e0-fd910b34dcbc-maas-api-service-ca\") pod \"f1729b14-3267-442f-88e0-fd910b34dcbc\" (UID: \"f1729b14-3267-442f-88e0-fd910b34dcbc\") "
Sep 06 19:38:34.584773 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:34.584750    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1729b14-3267-442f-88e0-fd910b34dcbc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f1729b14-3267-442f-88e0-fd910b34dcbc" (UID: "f1729b14-3267-442f-88e0-fd910b34dcbc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:38:34.584901 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:34.584888    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1729b14-3267-442f-88e0-fd910b34dcbc-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:35.269391 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:35.269357    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"
Sep 06 19:38:35.269575 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:35.269393    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s" event={"ID":"f1729b14-3267-442f-88e0-fd910b34dcbc","Type":"ContainerDied","Data":"71dba3b60d12fa73a33e4d14e302cf9282aaffad6b4628ed8ecc83e0a2047cab"}
Sep 06 19:38:35.269575 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:35.269441    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71dba3b60d12fa73a33e4d14e302cf9282aaffad6b4628ed8ecc83e0a2047cab"
Sep 06 19:38:36.070317 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.070252    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:38:36.202486 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202450    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcdcde74-66c4-4222-aca0-69db39e9163b-istiod-ca-cert\") pod \"bcdcde74-66c4-4222-aca0-69db39e9163b\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") "
Sep 06 19:38:36.202682 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202536    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-certs\") pod \"bcdcde74-66c4-4222-aca0-69db39e9163b\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") "
Sep 06 19:38:36.202682 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202565    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-podinfo\") pod \"bcdcde74-66c4-4222-aca0-69db39e9163b\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") "
Sep 06 19:38:36.202682 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202593    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6lnd\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-kube-api-access-r6lnd\") pod \"bcdcde74-66c4-4222-aca0-69db39e9163b\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") "
Sep 06 19:38:36.202682 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202642    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-data\") pod \"bcdcde74-66c4-4222-aca0-69db39e9163b\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") "
Sep 06 19:38:36.202682 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202682    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-envoy\") pod \"bcdcde74-66c4-4222-aca0-69db39e9163b\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") "
Sep 06 19:38:36.202951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202726    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-socket\") pod \"bcdcde74-66c4-4222-aca0-69db39e9163b\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") "
Sep 06 19:38:36.202951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202760    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-credential-socket\") pod \"bcdcde74-66c4-4222-aca0-69db39e9163b\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") "
Sep 06 19:38:36.202951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202790    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-token\") pod \"bcdcde74-66c4-4222-aca0-69db39e9163b\" (UID: \"bcdcde74-66c4-4222-aca0-69db39e9163b\") "
Sep 06 19:38:36.202951 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202879    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcdcde74-66c4-4222-aca0-69db39e9163b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bcdcde74-66c4-4222-aca0-69db39e9163b" (UID: "bcdcde74-66c4-4222-aca0-69db39e9163b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:38:36.203246 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202891    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bcdcde74-66c4-4222-aca0-69db39e9163b" (UID: "bcdcde74-66c4-4222-aca0-69db39e9163b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:36.203246 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.202979    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-data" (OuterVolumeSpecName: "istio-data") pod "bcdcde74-66c4-4222-aca0-69db39e9163b" (UID: "bcdcde74-66c4-4222-aca0-69db39e9163b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:36.203246 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.203121    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bcdcde74-66c4-4222-aca0-69db39e9163b" (UID: "bcdcde74-66c4-4222-aca0-69db39e9163b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:36.203246 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.203220    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bcdcde74-66c4-4222-aca0-69db39e9163b" (UID: "bcdcde74-66c4-4222-aca0-69db39e9163b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:36.203412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.203314    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:36.203412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.203334    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:36.203412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.203349    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcdcde74-66c4-4222-aca0-69db39e9163b-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:36.203412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.203365    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:36.203412 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.203378    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:36.205325 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.205296    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-token" (OuterVolumeSpecName: "istio-token") pod "bcdcde74-66c4-4222-aca0-69db39e9163b" (UID: "bcdcde74-66c4-4222-aca0-69db39e9163b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:38:36.205432 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.205297    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bcdcde74-66c4-4222-aca0-69db39e9163b" (UID: "bcdcde74-66c4-4222-aca0-69db39e9163b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:36.205539 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.205512    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-kube-api-access-r6lnd" (OuterVolumeSpecName: "kube-api-access-r6lnd") pod "bcdcde74-66c4-4222-aca0-69db39e9163b" (UID: "bcdcde74-66c4-4222-aca0-69db39e9163b"). InnerVolumeSpecName "kube-api-access-r6lnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:38:36.205601 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.205543    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bcdcde74-66c4-4222-aca0-69db39e9163b" (UID: "bcdcde74-66c4-4222-aca0-69db39e9163b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:38:36.279901 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.279852    2581 generic.go:356] "Generic (PLEG): container finished" podID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerID="22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7" exitCode=0
Sep 06 19:38:36.280128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.280009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" event={"ID":"bcdcde74-66c4-4222-aca0-69db39e9163b","Type":"ContainerDied","Data":"22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7"}
Sep 06 19:38:36.280128 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.280050    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"
Sep 06 19:38:36.280266 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.280123    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv" event={"ID":"bcdcde74-66c4-4222-aca0-69db39e9163b","Type":"ContainerDied","Data":"2a9adbfb3e19ef036859726e6aac35dfa46038a78aed240f7d1546344e708313"}
Sep 06 19:38:36.280266 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.280166    2581 scope.go:117] "RemoveContainer" containerID="22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7"
Sep 06 19:38:36.290433 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.290395    2581 scope.go:117] "RemoveContainer" containerID="5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef"
Sep 06 19:38:36.297019 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.296990    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"]
Sep 06 19:38:36.300709 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.300688    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c31435-openshift-default-668ff779f8-gszhv"]
Sep 06 19:38:36.300780 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.300746    2581 scope.go:117] "RemoveContainer" containerID="22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7"
Sep 06 19:38:36.301113 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:38:36.301094    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7\": container with ID starting with 22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7 not found: ID does not exist" containerID="22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7"
Sep 06 19:38:36.301193 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.301126    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7"} err="failed to get container status \"22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7\": rpc error: code = NotFound desc = could not find container \"22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7\": container with ID starting with 22417c6078de8dd066e6542788c30e3d958789504dcf36a1317131633181b9d7 not found: ID does not exist"
Sep 06 19:38:36.301193 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.301152    2581 scope.go:117] "RemoveContainer" containerID="5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef"
Sep 06 19:38:36.301475 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:38:36.301452    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef\": container with ID starting with 5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef not found: ID does not exist" containerID="5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef"
Sep 06 19:38:36.301547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.301502    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef"} err="failed to get container status \"5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef\": rpc error: code = NotFound desc = could not find container \"5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef\": container with ID starting with 5ce9f08879103b8f039b8b3965198bfbdb62a4d4d058531dcf6a5520da7a93ef not found: ID does not exist"
Sep 06 19:38:36.303941 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.303923    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:36.303992 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.303945    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:36.303992 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.303953    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcdcde74-66c4-4222-aca0-69db39e9163b-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:36.303992 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:36.303961    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6lnd\" (UniqueName: \"kubernetes.io/projected/bcdcde74-66c4-4222-aca0-69db39e9163b-kube-api-access-r6lnd\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:38.266593 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:38.266556    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" path="/var/lib/kubelet/pods/bcdcde74-66c4-4222-aca0-69db39e9163b/volumes"
Sep 06 19:38:40.749037 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:40.749000    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:43.804308 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:43.804277    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:44.727873 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:44.727834    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"]
Sep 06 19:38:44.751080 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:44.751038    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:46.288464 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:46.288433    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:46.388364 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:46.388320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:46.453330 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:46.453294    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"]
Sep 06 19:38:46.454714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:46.454683    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:46.455859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:46.455837    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-49a2d2-zvn5s"]
Sep 06 19:38:46.504690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:46.504652    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:46.802574 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:46.802542    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:48.266572 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:48.266526    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1729b14-3267-442f-88e0-fd910b34dcbc" path="/var/lib/kubelet/pods/f1729b14-3267-442f-88e0-fd910b34dcbc/volumes"
Sep 06 19:38:51.641646 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:51.641608    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:52.567338 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:52.567299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:52.667643 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:52.667614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:52.790877 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:52.790845    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"]
Sep 06 19:38:52.791362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:52.791327    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy" containerID="cri-o://dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7" gracePeriod=30
Sep 06 19:38:52.805636 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:52.805602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:53.122325 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.122287    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"]
Sep 06 19:38:53.122750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.122737    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1729b14-3267-442f-88e0-fd910b34dcbc" containerName="revoke-keys"
Sep 06 19:38:53.122750 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.122751    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1729b14-3267-442f-88e0-fd910b34dcbc" containerName="revoke-keys"
Sep 06 19:38:53.122843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.122769    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy"
Sep 06 19:38:53.122843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.122775    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy"
Sep 06 19:38:53.122843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.122789    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy"
Sep 06 19:38:53.122843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.122794    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy"
Sep 06 19:38:53.122959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.122868    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy"
Sep 06 19:38:53.122959 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.122878    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1729b14-3267-442f-88e0-fd910b34dcbc" containerName="revoke-keys"
Sep 06 19:38:53.126331 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.126315    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"
Sep 06 19:38:53.128122 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.128097    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:38:53.128264 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.128211    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:38:53.132555 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.132528    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"]
Sep 06 19:38:53.183020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.182984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6043f92c-0b43-4515-a56b-1623653d1408-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk\" (UID: \"6043f92c-0b43-4515-a56b-1623653d1408\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"
Sep 06 19:38:53.219689 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.219649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:53.287102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.287040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6043f92c-0b43-4515-a56b-1623653d1408-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk\" (UID: \"6043f92c-0b43-4515-a56b-1623653d1408\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"
Sep 06 19:38:53.287866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.287839    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6043f92c-0b43-4515-a56b-1623653d1408-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk\" (UID: \"6043f92c-0b43-4515-a56b-1623653d1408\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"
Sep 06 19:38:53.440901 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.440866    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"
Sep 06 19:38:53.716282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:53.716259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"]
Sep 06 19:38:53.718453 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:38:53.718427    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6043f92c_0b43_4515_a56b_1623653d1408.slice/crio-7d41a86a96ba41ecada82973275fb59530e713e8f146ea0bf2e2f5b9c6b2bd1c WatchSource:0}: Error finding container 7d41a86a96ba41ecada82973275fb59530e713e8f146ea0bf2e2f5b9c6b2bd1c: Status 404 returned error can't find the container with id 7d41a86a96ba41ecada82973275fb59530e713e8f146ea0bf2e2f5b9c6b2bd1c
Sep 06 19:38:54.389735 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:54.389700    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk" event={"ID":"6043f92c-0b43-4515-a56b-1623653d1408","Type":"ContainerStarted","Data":"501717f1401fafe685549e7d11cd02ff0dcb3240abc1a82fb3f85612da561261"}
Sep 06 19:38:54.389916 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:54.389762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk" event={"ID":"6043f92c-0b43-4515-a56b-1623653d1408","Type":"ContainerStarted","Data":"7d41a86a96ba41ecada82973275fb59530e713e8f146ea0bf2e2f5b9c6b2bd1c"}
Sep 06 19:38:54.407181 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:54.407133    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk" podStartSLOduration=1.407118487 podStartE2EDuration="1.407118487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:38:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:38:54.404381187 +0000 UTC m=+1662.684833982" watchObservedRunningTime="2026-09-06 19:38:54.407118487 +0000 UTC m=+1662.687571280"
Sep 06 19:38:54.533968 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:54.533934    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:38:55.397876 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:55.397839    2581 generic.go:356] "Generic (PLEG): container finished" podID="6043f92c-0b43-4515-a56b-1623653d1408" containerID="501717f1401fafe685549e7d11cd02ff0dcb3240abc1a82fb3f85612da561261" exitCode=0
Sep 06 19:38:55.398375 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:55.397881    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk" event={"ID":"6043f92c-0b43-4515-a56b-1623653d1408","Type":"ContainerDied","Data":"501717f1401fafe685549e7d11cd02ff0dcb3240abc1a82fb3f85612da561261"}
Sep 06 19:38:56.545677 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:56.545650    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"
Sep 06 19:38:56.623481 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:56.623445    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6043f92c-0b43-4515-a56b-1623653d1408-maas-api-service-ca\") pod \"6043f92c-0b43-4515-a56b-1623653d1408\" (UID: \"6043f92c-0b43-4515-a56b-1623653d1408\") "
Sep 06 19:38:56.623769 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:56.623748    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6043f92c-0b43-4515-a56b-1623653d1408-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6043f92c-0b43-4515-a56b-1623653d1408" (UID: "6043f92c-0b43-4515-a56b-1623653d1408"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:38:56.725170 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:56.725083    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6043f92c-0b43-4515-a56b-1623653d1408-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:57.411973 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:57.411944    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"
Sep 06 19:38:57.411973 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:57.411969    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk" event={"ID":"6043f92c-0b43-4515-a56b-1623653d1408","Type":"ContainerDied","Data":"7d41a86a96ba41ecada82973275fb59530e713e8f146ea0bf2e2f5b9c6b2bd1c"}
Sep 06 19:38:57.412224 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:57.412011    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d41a86a96ba41ecada82973275fb59530e713e8f146ea0bf2e2f5b9c6b2bd1c"
Sep 06 19:38:58.063314 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.063243    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:38:58.139764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.139726    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bdb541c-74c1-484f-b7ba-449f05173a9d-istiod-ca-cert\") pod \"6bdb541c-74c1-484f-b7ba-449f05173a9d\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") "
Sep 06 19:38:58.139764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.139774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-socket\") pod \"6bdb541c-74c1-484f-b7ba-449f05173a9d\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") "
Sep 06 19:38:58.140026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.139794    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-credential-socket\") pod \"6bdb541c-74c1-484f-b7ba-449f05173a9d\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") "
Sep 06 19:38:58.140026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.139832    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-data\") pod \"6bdb541c-74c1-484f-b7ba-449f05173a9d\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") "
Sep 06 19:38:58.140026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.139867    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-certs\") pod \"6bdb541c-74c1-484f-b7ba-449f05173a9d\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") "
Sep 06 19:38:58.140026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.139900    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-envoy\") pod \"6bdb541c-74c1-484f-b7ba-449f05173a9d\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") "
Sep 06 19:38:58.140026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.139935    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rctj\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-kube-api-access-5rctj\") pod \"6bdb541c-74c1-484f-b7ba-449f05173a9d\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") "
Sep 06 19:38:58.140026 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.139990    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-podinfo\") pod \"6bdb541c-74c1-484f-b7ba-449f05173a9d\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") "
Sep 06 19:38:58.140394 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140048    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-token\") pod \"6bdb541c-74c1-484f-b7ba-449f05173a9d\" (UID: \"6bdb541c-74c1-484f-b7ba-449f05173a9d\") "
Sep 06 19:38:58.140394 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140085    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6bdb541c-74c1-484f-b7ba-449f05173a9d" (UID: "6bdb541c-74c1-484f-b7ba-449f05173a9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:58.140394 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140269    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6bdb541c-74c1-484f-b7ba-449f05173a9d" (UID: "6bdb541c-74c1-484f-b7ba-449f05173a9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:58.140394 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140288    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-data" (OuterVolumeSpecName: "istio-data") pod "6bdb541c-74c1-484f-b7ba-449f05173a9d" (UID: "6bdb541c-74c1-484f-b7ba-449f05173a9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:58.140394 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140350    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bdb541c-74c1-484f-b7ba-449f05173a9d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6bdb541c-74c1-484f-b7ba-449f05173a9d" (UID: "6bdb541c-74c1-484f-b7ba-449f05173a9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:38:58.140604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140442    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6bdb541c-74c1-484f-b7ba-449f05173a9d" (UID: "6bdb541c-74c1-484f-b7ba-449f05173a9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:58.140604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140527    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bdb541c-74c1-484f-b7ba-449f05173a9d-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:58.140604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140546    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:58.140604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140562    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:58.140604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140578    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:58.140604 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.140593    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:58.142332 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.142309    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bdb541c-74c1-484f-b7ba-449f05173a9d" (UID: "6bdb541c-74c1-484f-b7ba-449f05173a9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:38:58.142483 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.142342    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6bdb541c-74c1-484f-b7ba-449f05173a9d" (UID: "6bdb541c-74c1-484f-b7ba-449f05173a9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:38:58.142718 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.142692    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6bdb541c-74c1-484f-b7ba-449f05173a9d" (UID: "6bdb541c-74c1-484f-b7ba-449f05173a9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:38:58.142823 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.142737    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-kube-api-access-5rctj" (OuterVolumeSpecName: "kube-api-access-5rctj") pod "6bdb541c-74c1-484f-b7ba-449f05173a9d" (UID: "6bdb541c-74c1-484f-b7ba-449f05173a9d"). InnerVolumeSpecName "kube-api-access-5rctj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:38:58.241868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.241832    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:58.241868 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.241866    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rctj\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-kube-api-access-5rctj\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:58.242125 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.241877    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:58.242125 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.241886    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bdb541c-74c1-484f-b7ba-449f05173a9d-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:38:58.419399 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.419369    2581 generic.go:356] "Generic (PLEG): container finished" podID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerID="dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7" exitCode=0
Sep 06 19:38:58.419579 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.419419    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"
Sep 06 19:38:58.419579 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.419426    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" event={"ID":"6bdb541c-74c1-484f-b7ba-449f05173a9d","Type":"ContainerDied","Data":"dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7"}
Sep 06 19:38:58.419579 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.419469    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd" event={"ID":"6bdb541c-74c1-484f-b7ba-449f05173a9d","Type":"ContainerDied","Data":"a2abfd0fe520c4d94aa255eb2875dc6863a241a14db822ffa82fdcac5d6bdf71"}
Sep 06 19:38:58.419579 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.419491    2581 scope.go:117] "RemoveContainer" containerID="dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7"
Sep 06 19:38:58.428832 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.428810    2581 scope.go:117] "RemoveContainer" containerID="a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911"
Sep 06 19:38:58.435658 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.435636    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"]
Sep 06 19:38:58.438656 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.438639    2581 scope.go:117] "RemoveContainer" containerID="dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7"
Sep 06 19:38:58.438737 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.438710    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-49a2d2-openshift-default-97d778d86-tg9wd"]
Sep 06 19:38:58.438967 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:38:58.438949    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7\": container with ID starting with dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7 not found: ID does not exist" containerID="dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7"
Sep 06 19:38:58.439014 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.438976    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7"} err="failed to get container status \"dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7\": rpc error: code = NotFound desc = could not find container \"dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7\": container with ID starting with dd3f33d5bbc16dcabce1c9e5ecc1285f800113c3d04ca38a622537c962376ec7 not found: ID does not exist"
Sep 06 19:38:58.439014 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.438994    2581 scope.go:117] "RemoveContainer" containerID="a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911"
Sep 06 19:38:58.439284 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:38:58.439266    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911\": container with ID starting with a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911 not found: ID does not exist" containerID="a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911"
Sep 06 19:38:58.439335 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:58.439293    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911"} err="failed to get container status \"a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911\": rpc error: code = NotFound desc = could not find container \"a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911\": container with ID starting with a9acd2c6d9b59b44b570a0a9c33c74cc8822605386f7e6e72c8c709bd9a57911 not found: ID does not exist"
Sep 06 19:38:59.750046 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:38:59.749998    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56" podUID="384b1dfe-b680-4ace-a2db-b3b48bbe1006" containerName="main" containerID="cri-o://9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297" gracePeriod=45
Sep 06 19:39:00.010462 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.010374    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:39:00.161705 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.161673    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tls-certs\") pod \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") "
Sep 06 19:39:00.161896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.161713    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-dshm\") pod \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") "
Sep 06 19:39:00.161896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.161752    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-model-cache\") pod \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") "
Sep 06 19:39:00.161896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.161811    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-home\") pod \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") "
Sep 06 19:39:00.161896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.161848    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp8qz\" (UniqueName: \"kubernetes.io/projected/384b1dfe-b680-4ace-a2db-b3b48bbe1006-kube-api-access-bp8qz\") pod \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") "
Sep 06 19:39:00.162145 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.162005    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tmp-dir\") pod \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\" (UID: \"384b1dfe-b680-4ace-a2db-b3b48bbe1006\") "
Sep 06 19:39:00.162145 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.162080    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-home" (OuterVolumeSpecName: "home") pod "384b1dfe-b680-4ace-a2db-b3b48bbe1006" (UID: "384b1dfe-b680-4ace-a2db-b3b48bbe1006"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:00.162145 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.162098    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-model-cache" (OuterVolumeSpecName: "model-cache") pod "384b1dfe-b680-4ace-a2db-b3b48bbe1006" (UID: "384b1dfe-b680-4ace-a2db-b3b48bbe1006"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:00.162307 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.162287    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "384b1dfe-b680-4ace-a2db-b3b48bbe1006" (UID: "384b1dfe-b680-4ace-a2db-b3b48bbe1006"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:00.162356 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.162346    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-model-cache\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:00.162392 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.162359    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-home\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:00.164096 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.164046    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-dshm" (OuterVolumeSpecName: "dshm") pod "384b1dfe-b680-4ace-a2db-b3b48bbe1006" (UID: "384b1dfe-b680-4ace-a2db-b3b48bbe1006"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:00.164096 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.164052    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "384b1dfe-b680-4ace-a2db-b3b48bbe1006" (UID: "384b1dfe-b680-4ace-a2db-b3b48bbe1006"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 19:39:00.164234 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.164124    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/384b1dfe-b680-4ace-a2db-b3b48bbe1006-kube-api-access-bp8qz" (OuterVolumeSpecName: "kube-api-access-bp8qz") pod "384b1dfe-b680-4ace-a2db-b3b48bbe1006" (UID: "384b1dfe-b680-4ace-a2db-b3b48bbe1006"). InnerVolumeSpecName "kube-api-access-bp8qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:39:00.263489 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.263390    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tmp-dir\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:00.263489 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.263420    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/384b1dfe-b680-4ace-a2db-b3b48bbe1006-tls-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:00.263489 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.263435    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/384b1dfe-b680-4ace-a2db-b3b48bbe1006-dshm\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:00.263489 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.263448    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp8qz\" (UniqueName: \"kubernetes.io/projected/384b1dfe-b680-4ace-a2db-b3b48bbe1006-kube-api-access-bp8qz\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:00.266445 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.266403    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" path="/var/lib/kubelet/pods/6bdb541c-74c1-484f-b7ba-449f05173a9d/volumes"
Sep 06 19:39:00.432593 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.432555    2581 generic.go:356] "Generic (PLEG): container finished" podID="384b1dfe-b680-4ace-a2db-b3b48bbe1006" containerID="9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297" exitCode=0
Sep 06 19:39:00.432796 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.432603    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"
Sep 06 19:39:00.432796 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.432618    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56" event={"ID":"384b1dfe-b680-4ace-a2db-b3b48bbe1006","Type":"ContainerDied","Data":"9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297"}
Sep 06 19:39:00.432796 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.432659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56" event={"ID":"384b1dfe-b680-4ace-a2db-b3b48bbe1006","Type":"ContainerDied","Data":"2152b64edccb81d1a8e65aacb0b3bae35377c1729489d4bb522ba014b8a7f67e"}
Sep 06 19:39:00.432796 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.432680    2581 scope.go:117] "RemoveContainer" containerID="9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297"
Sep 06 19:39:00.441681 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.441665    2581 scope.go:117] "RemoveContainer" containerID="9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297"
Sep 06 19:39:00.441933 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:39:00.441915    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297\": container with ID starting with 9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297 not found: ID does not exist" containerID="9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297"
Sep 06 19:39:00.441978 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.441940    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297"} err="failed to get container status \"9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297\": rpc error: code = NotFound desc = could not find container \"9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297\": container with ID starting with 9073f34a6384193107cc830f65a2effcbab7936d2b242699a43bc4a43ff31297 not found: ID does not exist"
Sep 06 19:39:00.447333 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.447308    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"]
Sep 06 19:39:00.450518 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:00.450496    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-476f98/sub-test-model-476f98-kserve-6c9d97db99-tcn56"]
Sep 06 19:39:02.266268 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:02.266231    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="384b1dfe-b680-4ace-a2db-b3b48bbe1006" path="/var/lib/kubelet/pods/384b1dfe-b680-4ace-a2db-b3b48bbe1006/volumes"
Sep 06 19:39:03.232366 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:03.232311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:03.814456 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:03.814419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:03.887406 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:03.887328    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:03.942250 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:03.942201    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:03.993656 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:03.993618    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"]
Sep 06 19:39:03.996286 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:03.996254    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-cf729a-rf6vk"]
Sep 06 19:39:04.000143 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:04.000119    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:04.266377 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:04.266344    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6043f92c-0b43-4515-a56b-1623653d1408" path="/var/lib/kubelet/pods/6043f92c-0b43-4515-a56b-1623653d1408/volumes"
Sep 06 19:39:04.331388 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:04.331361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:09.916815 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:09.916780    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:09.973637 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:09.973602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:10.149841 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.149804    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"]
Sep 06 19:39:10.150118 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.150089    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy" containerID="cri-o://7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273" gracePeriod=30
Sep 06 19:39:10.167260 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.167181    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:10.487104 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.486988    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"]
Sep 06 19:39:10.487487 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487468    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy"
Sep 06 19:39:10.487568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487489    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy"
Sep 06 19:39:10.487568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487502    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="384b1dfe-b680-4ace-a2db-b3b48bbe1006" containerName="main"
Sep 06 19:39:10.487568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487508    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="384b1dfe-b680-4ace-a2db-b3b48bbe1006" containerName="main"
Sep 06 19:39:10.487568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487530    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6043f92c-0b43-4515-a56b-1623653d1408" containerName="revoke-keys"
Sep 06 19:39:10.487568 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487535    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6043f92c-0b43-4515-a56b-1623653d1408" containerName="revoke-keys"
Sep 06 19:39:10.487827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487603    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="384b1dfe-b680-4ace-a2db-b3b48bbe1006" containerName="main"
Sep 06 19:39:10.487827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487612    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy"
Sep 06 19:39:10.487827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487620    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy"
Sep 06 19:39:10.487827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487626    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcdcde74-66c4-4222-aca0-69db39e9163b" containerName="istio-proxy"
Sep 06 19:39:10.487827 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.487636    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6043f92c-0b43-4515-a56b-1623653d1408" containerName="revoke-keys"
Sep 06 19:39:10.490850 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.490830    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"
Sep 06 19:39:10.493231 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.493208    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:39:10.493355 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.493244    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:39:10.503809 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.503774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"]
Sep 06 19:39:10.581004 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.580966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:10.679155 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.679104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c98053-1af3-4ac1-a85d-2b466662fe6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg\" (UID: \"88c98053-1af3-4ac1-a85d-2b466662fe6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"
Sep 06 19:39:10.780695 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.780591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c98053-1af3-4ac1-a85d-2b466662fe6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg\" (UID: \"88c98053-1af3-4ac1-a85d-2b466662fe6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"
Sep 06 19:39:10.781337 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.781313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c98053-1af3-4ac1-a85d-2b466662fe6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg\" (UID: \"88c98053-1af3-4ac1-a85d-2b466662fe6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"
Sep 06 19:39:10.810503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:10.810465    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"
Sep 06 19:39:11.086105 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:11.086018    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"]
Sep 06 19:39:11.087945 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:39:11.087916    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c98053_1af3_4ac1_a85d_2b466662fe6d.slice/crio-695cda7c4df6595f1782fe3f06a8c7f534da5426a74964ca634240247b7e28ac WatchSource:0}: Error finding container 695cda7c4df6595f1782fe3f06a8c7f534da5426a74964ca634240247b7e28ac: Status 404 returned error can't find the container with id 695cda7c4df6595f1782fe3f06a8c7f534da5426a74964ca634240247b7e28ac
Sep 06 19:39:11.514623 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:11.514586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg" event={"ID":"88c98053-1af3-4ac1-a85d-2b466662fe6d","Type":"ContainerStarted","Data":"cb3c0b1aef18577576ff5711395d9963966e1648e00fd69fcd886598d4b955fa"}
Sep 06 19:39:11.514810 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:11.514639    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg" event={"ID":"88c98053-1af3-4ac1-a85d-2b466662fe6d","Type":"ContainerStarted","Data":"695cda7c4df6595f1782fe3f06a8c7f534da5426a74964ca634240247b7e28ac"}
Sep 06 19:39:11.529584 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:11.529532    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg" podStartSLOduration=1.529513007 podStartE2EDuration="1.529513007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:39:11.526509871 +0000 UTC m=+1679.806962670" watchObservedRunningTime="2026-09-06 19:39:11.529513007 +0000 UTC m=+1679.809965801"
Sep 06 19:39:12.522857 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:12.522820    2581 generic.go:356] "Generic (PLEG): container finished" podID="88c98053-1af3-4ac1-a85d-2b466662fe6d" containerID="cb3c0b1aef18577576ff5711395d9963966e1648e00fd69fcd886598d4b955fa" exitCode=0
Sep 06 19:39:12.523328 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:12.522928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg" event={"ID":"88c98053-1af3-4ac1-a85d-2b466662fe6d","Type":"ContainerDied","Data":"cb3c0b1aef18577576ff5711395d9963966e1648e00fd69fcd886598d4b955fa"}
Sep 06 19:39:12.580954 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:12.580916    2581 scope.go:117] "RemoveContainer" containerID="a8357ae7f25d52ae9a827f5fa8a2d6aa40ebddfa4c2e9a03419034340a4d3e06"
Sep 06 19:39:12.591637 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:12.591611    2581 scope.go:117] "RemoveContainer" containerID="beda8d46f44a79b50049aef8a3667de79f795433bdae3bdc3b6e13bbee704c75"
Sep 06 19:39:12.602310 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:12.602283    2581 scope.go:117] "RemoveContainer" containerID="7c5e6039f3ede8b195b56af177d2c3066c8c5e4c02939ab8afc61fbab30fc9e5"
Sep 06 19:39:13.685880 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:13.685846    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"
Sep 06 19:39:13.709455 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:13.709423    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c98053-1af3-4ac1-a85d-2b466662fe6d-maas-api-service-ca\") pod \"88c98053-1af3-4ac1-a85d-2b466662fe6d\" (UID: \"88c98053-1af3-4ac1-a85d-2b466662fe6d\") "
Sep 06 19:39:13.709780 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:13.709757    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88c98053-1af3-4ac1-a85d-2b466662fe6d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "88c98053-1af3-4ac1-a85d-2b466662fe6d" (UID: "88c98053-1af3-4ac1-a85d-2b466662fe6d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:39:13.709882 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:13.709856    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c98053-1af3-4ac1-a85d-2b466662fe6d-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:14.535843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:14.535754    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"
Sep 06 19:39:14.535843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:14.535788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg" event={"ID":"88c98053-1af3-4ac1-a85d-2b466662fe6d","Type":"ContainerDied","Data":"695cda7c4df6595f1782fe3f06a8c7f534da5426a74964ca634240247b7e28ac"}
Sep 06 19:39:14.535843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:14.535842    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="695cda7c4df6595f1782fe3f06a8c7f534da5426a74964ca634240247b7e28ac"
Sep 06 19:39:15.119844 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.119800    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 19:39:15.120276 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.119865    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 19:39:15.410969 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.410942    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:39:15.427370 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427337    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-data\") pod \"8819cce4-d25e-41f7-8ef2-a0732178849f\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") "
Sep 06 19:39:15.427586 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427383    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-certs\") pod \"8819cce4-d25e-41f7-8ef2-a0732178849f\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") "
Sep 06 19:39:15.427586 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427526    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-podinfo\") pod \"8819cce4-d25e-41f7-8ef2-a0732178849f\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") "
Sep 06 19:39:15.427726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427591    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-envoy\") pod \"8819cce4-d25e-41f7-8ef2-a0732178849f\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") "
Sep 06 19:39:15.427726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427622    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-token\") pod \"8819cce4-d25e-41f7-8ef2-a0732178849f\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") "
Sep 06 19:39:15.427726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427651    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-credential-socket\") pod \"8819cce4-d25e-41f7-8ef2-a0732178849f\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") "
Sep 06 19:39:15.427726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427650    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-data" (OuterVolumeSpecName: "istio-data") pod "8819cce4-d25e-41f7-8ef2-a0732178849f" (UID: "8819cce4-d25e-41f7-8ef2-a0732178849f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:15.427726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427686    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8819cce4-d25e-41f7-8ef2-a0732178849f-istiod-ca-cert\") pod \"8819cce4-d25e-41f7-8ef2-a0732178849f\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") "
Sep 06 19:39:15.427726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427725    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n44s7\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-kube-api-access-n44s7\") pod \"8819cce4-d25e-41f7-8ef2-a0732178849f\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") "
Sep 06 19:39:15.427726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.427753    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-socket\") pod \"8819cce4-d25e-41f7-8ef2-a0732178849f\" (UID: \"8819cce4-d25e-41f7-8ef2-a0732178849f\") "
Sep 06 19:39:15.428205 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.428182    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:15.428669 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.428505    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8819cce4-d25e-41f7-8ef2-a0732178849f" (UID: "8819cce4-d25e-41f7-8ef2-a0732178849f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:15.428669 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.428579    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8819cce4-d25e-41f7-8ef2-a0732178849f" (UID: "8819cce4-d25e-41f7-8ef2-a0732178849f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:15.428989 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.428790    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8819cce4-d25e-41f7-8ef2-a0732178849f" (UID: "8819cce4-d25e-41f7-8ef2-a0732178849f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:15.428989 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.428876    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8819cce4-d25e-41f7-8ef2-a0732178849f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8819cce4-d25e-41f7-8ef2-a0732178849f" (UID: "8819cce4-d25e-41f7-8ef2-a0732178849f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:39:15.431408 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.431001    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8819cce4-d25e-41f7-8ef2-a0732178849f" (UID: "8819cce4-d25e-41f7-8ef2-a0732178849f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:39:15.431860 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.431807    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-token" (OuterVolumeSpecName: "istio-token") pod "8819cce4-d25e-41f7-8ef2-a0732178849f" (UID: "8819cce4-d25e-41f7-8ef2-a0732178849f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:39:15.431860 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.431822    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-kube-api-access-n44s7" (OuterVolumeSpecName: "kube-api-access-n44s7") pod "8819cce4-d25e-41f7-8ef2-a0732178849f" (UID: "8819cce4-d25e-41f7-8ef2-a0732178849f"). InnerVolumeSpecName "kube-api-access-n44s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:39:15.432221 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.432199    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8819cce4-d25e-41f7-8ef2-a0732178849f" (UID: "8819cce4-d25e-41f7-8ef2-a0732178849f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:15.529694 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.529657    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:15.529694 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.529687    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:15.529694 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.529697    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:15.529694 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.529706    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:15.529966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.529716    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8819cce4-d25e-41f7-8ef2-a0732178849f-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:15.529966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.529724    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n44s7\" (UniqueName: \"kubernetes.io/projected/8819cce4-d25e-41f7-8ef2-a0732178849f-kube-api-access-n44s7\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:15.529966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.529732    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:15.529966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.529740    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8819cce4-d25e-41f7-8ef2-a0732178849f-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:15.542947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.542911    2581 generic.go:356] "Generic (PLEG): container finished" podID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerID="7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273" exitCode=0
Sep 06 19:39:15.543136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.542961    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"
Sep 06 19:39:15.543136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.542994    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" event={"ID":"8819cce4-d25e-41f7-8ef2-a0732178849f","Type":"ContainerDied","Data":"7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273"}
Sep 06 19:39:15.543136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.543049    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k" event={"ID":"8819cce4-d25e-41f7-8ef2-a0732178849f","Type":"ContainerDied","Data":"9bb013688ee12aa7fb97d1908adaa22977dddf421304ece02995e68f422d171e"}
Sep 06 19:39:15.543136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.543103    2581 scope.go:117] "RemoveContainer" containerID="7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273"
Sep 06 19:39:15.552846 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.552828    2581 scope.go:117] "RemoveContainer" containerID="05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc"
Sep 06 19:39:15.561806 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.561792    2581 scope.go:117] "RemoveContainer" containerID="7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273"
Sep 06 19:39:15.562149 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:39:15.562125    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273\": container with ID starting with 7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273 not found: ID does not exist" containerID="7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273"
Sep 06 19:39:15.562256 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.562156    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273"} err="failed to get container status \"7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273\": rpc error: code = NotFound desc = could not find container \"7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273\": container with ID starting with 7daf97e26d4b0b7d723165c469fd300921d08bcbbe07cb0ab79ef329b00c1273 not found: ID does not exist"
Sep 06 19:39:15.562256 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.562174    2581 scope.go:117] "RemoveContainer" containerID="05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc"
Sep 06 19:39:15.562452 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:39:15.562435    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc\": container with ID starting with 05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc not found: ID does not exist" containerID="05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc"
Sep 06 19:39:15.562499 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.562460    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc"} err="failed to get container status \"05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc\": rpc error: code = NotFound desc = could not find container \"05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc\": container with ID starting with 05f5d7d058b38d021ff4e36457aee6b8199551a59516d60b67fdf04cb6ba08cc not found: ID does not exist"
Sep 06 19:39:15.563355 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.563335    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"]
Sep 06 19:39:15.568870 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:15.568848    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf729a-openshift-default-54b685b5c8-jtf8k"]
Sep 06 19:39:16.266756 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:16.266713    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" path="/var/lib/kubelet/pods/8819cce4-d25e-41f7-8ef2-a0732178849f/volumes"
Sep 06 19:39:20.873417 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:20.873386    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:20.934954 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:20.934915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:21.000084 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:21.000034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:21.036987 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:21.036944    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"]
Sep 06 19:39:21.039881 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:21.039853    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6bbb53-4vwxg"]
Sep 06 19:39:21.052615 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:21.052582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:21.395662 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:21.395619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:22.271972 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:22.271934    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88c98053-1af3-4ac1-a85d-2b466662fe6d" path="/var/lib/kubelet/pods/88c98053-1af3-4ac1-a85d-2b466662fe6d/volumes"
Sep 06 19:39:23.074684 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:23.074649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:26.775194 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:26.775163    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:27.084306 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:27.084222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:27.226842 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:27.226810    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:27.356916 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:27.356816    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"]
Sep 06 19:39:27.357179 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:27.357135    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy" containerID="cri-o://c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4" gracePeriod=30
Sep 06 19:39:27.380220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:27.380183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:39:32.625633 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.625567    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:39:32.643866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.643831    2581 generic.go:356] "Generic (PLEG): container finished" podID="883854de-8b63-45da-9763-f73dea0cb887" containerID="c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4" exitCode=0
Sep 06 19:39:32.643866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.643885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" event={"ID":"883854de-8b63-45da-9763-f73dea0cb887","Type":"ContainerDied","Data":"c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4"}
Sep 06 19:39:32.644634 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.643959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks" event={"ID":"883854de-8b63-45da-9763-f73dea0cb887","Type":"ContainerDied","Data":"edb9d666050ecc52f7e74f30ba022270d7b459170023569197454bd14b4bf868"}
Sep 06 19:39:32.644634 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.643992    2581 scope.go:117] "RemoveContainer" containerID="c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4"
Sep 06 19:39:32.644634 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.644220    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"
Sep 06 19:39:32.654130 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.654111    2581 scope.go:117] "RemoveContainer" containerID="5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb"
Sep 06 19:39:32.663119 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.663099    2581 scope.go:117] "RemoveContainer" containerID="c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4"
Sep 06 19:39:32.663436 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:39:32.663413    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4\": container with ID starting with c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4 not found: ID does not exist" containerID="c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4"
Sep 06 19:39:32.663521 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.663451    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4"} err="failed to get container status \"c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4\": rpc error: code = NotFound desc = could not find container \"c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4\": container with ID starting with c126b99be3b6b21bd61da5e8fa8e3d6e20a758dcae8ad0b41e525ef6430331d4 not found: ID does not exist"
Sep 06 19:39:32.663521 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.663475    2581 scope.go:117] "RemoveContainer" containerID="5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb"
Sep 06 19:39:32.663796 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:39:32.663773    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb\": container with ID starting with 5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb not found: ID does not exist" containerID="5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb"
Sep 06 19:39:32.663843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.663806    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb"} err="failed to get container status \"5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb\": rpc error: code = NotFound desc = could not find container \"5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb\": container with ID starting with 5a4a8d2a57040608d46b380fceeb1e66557bed9aa2a984aed71d48d70f251ccb not found: ID does not exist"
Sep 06 19:39:32.704988 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.704952    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-istio-token\") pod \"883854de-8b63-45da-9763-f73dea0cb887\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") "
Sep 06 19:39:32.705220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.704995    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-credential-socket\") pod \"883854de-8b63-45da-9763-f73dea0cb887\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") "
Sep 06 19:39:32.705220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705041    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-socket\") pod \"883854de-8b63-45da-9763-f73dea0cb887\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") "
Sep 06 19:39:32.705220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705105    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883854de-8b63-45da-9763-f73dea0cb887-istio-podinfo\") pod \"883854de-8b63-45da-9763-f73dea0cb887\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") "
Sep 06 19:39:32.705220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705132    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-envoy\") pod \"883854de-8b63-45da-9763-f73dea0cb887\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") "
Sep 06 19:39:32.705220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705165    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv5xq\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-kube-api-access-dv5xq\") pod \"883854de-8b63-45da-9763-f73dea0cb887\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") "
Sep 06 19:39:32.705493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705249    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883854de-8b63-45da-9763-f73dea0cb887-istiod-ca-cert\") pod \"883854de-8b63-45da-9763-f73dea0cb887\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") "
Sep 06 19:39:32.705493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705274    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-certs\") pod \"883854de-8b63-45da-9763-f73dea0cb887\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") "
Sep 06 19:39:32.705493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705332    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-data\") pod \"883854de-8b63-45da-9763-f73dea0cb887\" (UID: \"883854de-8b63-45da-9763-f73dea0cb887\") "
Sep 06 19:39:32.705493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705358    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "883854de-8b63-45da-9763-f73dea0cb887" (UID: "883854de-8b63-45da-9763-f73dea0cb887"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:32.705493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705391    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "883854de-8b63-45da-9763-f73dea0cb887" (UID: "883854de-8b63-45da-9763-f73dea0cb887"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:32.705800 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705595    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "883854de-8b63-45da-9763-f73dea0cb887" (UID: "883854de-8b63-45da-9763-f73dea0cb887"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:32.705800 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705672    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/883854de-8b63-45da-9763-f73dea0cb887-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "883854de-8b63-45da-9763-f73dea0cb887" (UID: "883854de-8b63-45da-9763-f73dea0cb887"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:39:32.705908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705827    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-data" (OuterVolumeSpecName: "istio-data") pod "883854de-8b63-45da-9763-f73dea0cb887" (UID: "883854de-8b63-45da-9763-f73dea0cb887"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:32.705952 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705920    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883854de-8b63-45da-9763-f73dea0cb887-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:32.705952 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705940    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:32.706032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705953    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:32.706032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.705962    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:32.707911 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.707877    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "883854de-8b63-45da-9763-f73dea0cb887" (UID: "883854de-8b63-45da-9763-f73dea0cb887"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:39:32.707911 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.707892    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-istio-token" (OuterVolumeSpecName: "istio-token") pod "883854de-8b63-45da-9763-f73dea0cb887" (UID: "883854de-8b63-45da-9763-f73dea0cb887"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:39:32.708125 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.708012    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/883854de-8b63-45da-9763-f73dea0cb887-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "883854de-8b63-45da-9763-f73dea0cb887" (UID: "883854de-8b63-45da-9763-f73dea0cb887"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:39:32.708125 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.708029    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-kube-api-access-dv5xq" (OuterVolumeSpecName: "kube-api-access-dv5xq") pod "883854de-8b63-45da-9763-f73dea0cb887" (UID: "883854de-8b63-45da-9763-f73dea0cb887"). InnerVolumeSpecName "kube-api-access-dv5xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:39:32.807368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.807321    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883854de-8b63-45da-9763-f73dea0cb887-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:32.807368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.807359    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:32.807368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.807372    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv5xq\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-kube-api-access-dv5xq\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:32.807624 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.807386    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883854de-8b63-45da-9763-f73dea0cb887-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:32.807624 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.807398    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/883854de-8b63-45da-9763-f73dea0cb887-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:39:32.967229 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.967186    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"]
Sep 06 19:39:32.969270 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:32.969247    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6bbb53-openshift-default-76784bc8d8-zw4ks"]
Sep 06 19:39:34.266236 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:39:34.266200    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="883854de-8b63-45da-9763-f73dea0cb887" path="/var/lib/kubelet/pods/883854de-8b63-45da-9763-f73dea0cb887/volumes"
Sep 06 19:40:04.384323 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:04.384289    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:07.000932 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:07.000894    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:07.961962 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:07.961913    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:08.372049 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.371956    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:08.633323 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633294    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"]
Sep 06 19:40:08.633800 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633784    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy"
Sep 06 19:40:08.633867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633802    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy"
Sep 06 19:40:08.633867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633812    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy"
Sep 06 19:40:08.633867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633817    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdb541c-74c1-484f-b7ba-449f05173a9d" containerName="istio-proxy"
Sep 06 19:40:08.633867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633831    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy"
Sep 06 19:40:08.633867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633836    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy"
Sep 06 19:40:08.633867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633843    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy"
Sep 06 19:40:08.633867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633850    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy"
Sep 06 19:40:08.633867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633859    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88c98053-1af3-4ac1-a85d-2b466662fe6d" containerName="revoke-keys"
Sep 06 19:40:08.633867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633868    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88c98053-1af3-4ac1-a85d-2b466662fe6d" containerName="revoke-keys"
Sep 06 19:40:08.634225 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633879    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy"
Sep 06 19:40:08.634225 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633884    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy"
Sep 06 19:40:08.634225 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633967    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy"
Sep 06 19:40:08.634225 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633979    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88c98053-1af3-4ac1-a85d-2b466662fe6d" containerName="revoke-keys"
Sep 06 19:40:08.634225 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633985    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy"
Sep 06 19:40:08.634225 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.633991    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="883854de-8b63-45da-9763-f73dea0cb887" containerName="istio-proxy"
Sep 06 19:40:08.634225 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.634000    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8819cce4-d25e-41f7-8ef2-a0732178849f" containerName="istio-proxy"
Sep 06 19:40:08.637485 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.637466    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.639791 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.639759    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-410f4d-openshift-default-dockercfg-klmk8\""
Sep 06 19:40:08.648553 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.648467    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"]
Sep 06 19:40:08.657841 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.657816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:08.774622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.774582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-envoy\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.774622 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.774624    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58e676d4-0a47-4604-a4be-6f3158cce967-istiod-ca-cert\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.774847 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.774673    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-credential-socket\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.774847 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.774694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58e676d4-0a47-4604-a4be-6f3158cce967-istio-podinfo\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.774847 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.774763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-data\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.774847 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.774782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmj4\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-kube-api-access-cbmj4\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.774847 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.774800    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-certs\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.774847 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.774831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-socket\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.775045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.774880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-istio-token\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876334 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-socket\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876518 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-istio-token\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876518 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876387    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-envoy\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876518 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58e676d4-0a47-4604-a4be-6f3158cce967-istiod-ca-cert\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876518 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-credential-socket\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876518 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58e676d4-0a47-4604-a4be-6f3158cce967-istio-podinfo\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876784 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-data\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876784 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmj4\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-kube-api-access-cbmj4\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876784 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-certs\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876784 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876750    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-socket\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-credential-socket\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876933    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-data\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.876997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.876938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-certs\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.877177 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.877160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58e676d4-0a47-4604-a4be-6f3158cce967-istiod-ca-cert\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.879078 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.879040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-envoy\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.879274 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.879255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58e676d4-0a47-4604-a4be-6f3158cce967-istio-podinfo\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.885100 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.884985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-istio-token\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.885233 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.885210    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmj4\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-kube-api-access-cbmj4\") pod \"e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") " pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:08.951996 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:08.951957    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:09.231478 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:09.231450    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"]
Sep 06 19:40:09.233151 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:40:09.233124    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58e676d4_0a47_4604_a4be_6f3158cce967.slice/crio-d668b9d002b5501210fdf23a4f81a9a34d5cd99ad51333f8deeb0b64f19f785e WatchSource:0}: Error finding container d668b9d002b5501210fdf23a4f81a9a34d5cd99ad51333f8deeb0b64f19f785e: Status 404 returned error can't find the container with id d668b9d002b5501210fdf23a4f81a9a34d5cd99ad51333f8deeb0b64f19f785e
Sep 06 19:40:09.235346 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:09.235316    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:09.235424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:09.235382    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:09.235424 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:09.235408    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:09.339855 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:09.339816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:09.861422 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:09.861379    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" event={"ID":"58e676d4-0a47-4604-a4be-6f3158cce967","Type":"ContainerStarted","Data":"00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3"}
Sep 06 19:40:09.861923 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:09.861443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" event={"ID":"58e676d4-0a47-4604-a4be-6f3158cce967","Type":"ContainerStarted","Data":"d668b9d002b5501210fdf23a4f81a9a34d5cd99ad51333f8deeb0b64f19f785e"}
Sep 06 19:40:09.880752 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:09.880698    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" podStartSLOduration=1.880679088 podStartE2EDuration="1.880679088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:40:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:40:09.87909154 +0000 UTC m=+1738.159544327" watchObservedRunningTime="2026-09-06 19:40:09.880679088 +0000 UTC m=+1738.161131882"
Sep 06 19:40:09.952151 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:09.952117    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:10.958463 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:10.958433    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:11.750214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:11.750176    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:11.848984 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:11.848953    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:11.872837 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:11.872808    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:11.873877 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:11.873859    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:12.886193 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:12.886165    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:12.987200 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:12.987113    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:13.090371 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:13.090294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:13.343535 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:13.343442    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:13.445989 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:13.445953    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:18.578452 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:18.578415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:18.628883 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:18.628841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:26.874611 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:26.874576    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 06 19:40:26.875045 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:26.874654    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 06 19:40:26.962733 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:26.962644    2581 generic.go:356] "Generic (PLEG): container finished" podID="58e676d4-0a47-4604-a4be-6f3158cce967" containerID="00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3" exitCode=0
Sep 06 19:40:26.962733 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:26.962680    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" event={"ID":"58e676d4-0a47-4604-a4be-6f3158cce967","Type":"ContainerDied","Data":"00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3"}
Sep 06 19:40:26.963134 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:26.963107    2581 scope.go:117] "RemoveContainer" containerID="00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3"
Sep 06 19:40:26.964215 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:26.964182    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:26.964316 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:26.964257    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:26.964316 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:26.964297    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:27.971161 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:27.971117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" event={"ID":"58e676d4-0a47-4604-a4be-6f3158cce967","Type":"ContainerStarted","Data":"31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22"}
Sep 06 19:40:28.952452 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:28.952406    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:28.952685 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:28.952662    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 06 19:40:28.952749 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:28.952729    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 06 19:40:29.952368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:29.952329    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 06 19:40:29.952769 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:29.952388    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 06 19:40:30.953021 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:30.952984    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 06 19:40:30.953445 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:30.953045    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 06 19:40:31.952781 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:31.952735    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 06 19:40:31.953002 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:31.952799    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 06 19:40:32.952954 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:32.952915    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 06 19:40:32.953383 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:32.953010    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 06 19:40:33.956862 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:33.956826    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:33.957275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:33.957190    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:33.957857 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:33.957836    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:40:36.708362 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:36.708329    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:37.332457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.332409    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:37.601332 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.601229    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"]
Sep 06 19:40:37.605176 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.605155    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.607259 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.607238    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-88ac22-openshift-default-dockercfg-mhzg7\""
Sep 06 19:40:37.614962 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.614938    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"]
Sep 06 19:40:37.641815 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.641777    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:37.678158 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.678122    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-data\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.678368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.678250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-socket\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.678368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.678298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf90734-d656-45f6-a506-2928819dd05b-istio-podinfo\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.678368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.678350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfjg7\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-kube-api-access-lfjg7\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.678368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.678366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf90734-d656-45f6-a506-2928819dd05b-istiod-ca-cert\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.678552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.678414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-istio-token\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.678552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.678467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-credential-socket\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.678552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.678483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-certs\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.678552 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.678506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-envoy\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.727124 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.727093    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:37.779091 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf90734-d656-45f6-a506-2928819dd05b-istiod-ca-cert\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-istio-token\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-credential-socket\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-certs\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-envoy\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779266    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-data\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779350    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-socket\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf90734-d656-45f6-a506-2928819dd05b-istio-podinfo\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfjg7\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-kube-api-access-lfjg7\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779767 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779625    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-credential-socket\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779767 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-certs\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779749    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-data\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-socket\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.779902 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.779862    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf90734-d656-45f6-a506-2928819dd05b-istiod-ca-cert\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.781677 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.781650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-envoy\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.781952 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.781936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf90734-d656-45f6-a506-2928819dd05b-istio-podinfo\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.786497 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.786467    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfjg7\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-kube-api-access-lfjg7\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.786660 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.786642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-istio-token\") pod \"e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") " pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:37.917017 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:37.916978    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:38.195732 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:38.195700    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"]
Sep 06 19:40:38.198433 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:40:38.198394    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf90734_d656_45f6_a506_2928819dd05b.slice/crio-b9cea15b8288d104e5fea7b12c8722784fc0493c5b6ccb478767138949fd81c1 WatchSource:0}: Error finding container b9cea15b8288d104e5fea7b12c8722784fc0493c5b6ccb478767138949fd81c1: Status 404 returned error can't find the container with id b9cea15b8288d104e5fea7b12c8722784fc0493c5b6ccb478767138949fd81c1
Sep 06 19:40:38.201252 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:38.201218    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:38.201337 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:38.201286    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:38.201337 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:38.201316    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:39.057755 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:39.057722    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" event={"ID":"3bf90734-d656-45f6-a506-2928819dd05b","Type":"ContainerStarted","Data":"0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f"}
Sep 06 19:40:39.058249 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:39.057778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" event={"ID":"3bf90734-d656-45f6-a506-2928819dd05b","Type":"ContainerStarted","Data":"b9cea15b8288d104e5fea7b12c8722784fc0493c5b6ccb478767138949fd81c1"}
Sep 06 19:40:39.076404 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:39.076354    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" podStartSLOduration=2.076336456 podStartE2EDuration="2.076336456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:40:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:40:39.075316194 +0000 UTC m=+1767.355768989" watchObservedRunningTime="2026-09-06 19:40:39.076336456 +0000 UTC m=+1767.356789250"
Sep 06 19:40:39.917296 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:39.917258    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:39.922390 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:39.922364    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:40.063925 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:40.063896    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:40.065037 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:40.065017    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:41.048987 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:41.048951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:41.150090 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:41.150031    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:42.191366 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:42.191335    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:42.296745 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:42.296707    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:42.470469 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:42.470361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:42.751676 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:42.751583    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:42.851616 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:42.851584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:40:56.106290 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:56.106229    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 19:40:56.106822 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:56.106317    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 19:40:58.172644 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:58.172610    2581 generic.go:356] "Generic (PLEG): container finished" podID="3bf90734-d656-45f6-a506-2928819dd05b" containerID="0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f" exitCode=0
Sep 06 19:40:58.173057 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:58.172687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" event={"ID":"3bf90734-d656-45f6-a506-2928819dd05b","Type":"ContainerDied","Data":"0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f"}
Sep 06 19:40:58.173146 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:58.173132    2581 scope.go:117] "RemoveContainer" containerID="0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f"
Sep 06 19:40:58.174368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:58.174338    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:58.174450 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:58.174434    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:58.174490 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:58.174473    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:40:59.180701 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:59.180663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" event={"ID":"3bf90734-d656-45f6-a506-2928819dd05b","Type":"ContainerStarted","Data":"ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d"}
Sep 06 19:40:59.917992 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:59.917949    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:40:59.918220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:59.918203    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 06 19:40:59.918278 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:40:59.918250    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 06 19:41:00.917630 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:00.917588    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 06 19:41:00.918025 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:00.917649    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 06 19:41:01.917977 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:01.917935    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 06 19:41:01.918423 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:01.918009    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 06 19:41:02.918342 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:02.918301    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 06 19:41:02.918745 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:02.918388    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 06 19:41:03.917586 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:03.917550    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 06 19:41:03.917783 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:03.917613    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 06 19:41:04.920969 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:04.920933    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:41:04.921354 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:04.921325    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:41:04.921998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:04.921977    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:41:05.912874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:05.912839    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:12.391600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:12.391570    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:41:12.398036 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:12.398010    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:41:12.400545 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:12.400523    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:41:12.406470 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:12.406451    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:41:12.666195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:12.666114    2581 scope.go:117] "RemoveContainer" containerID="e9cb566434f4303f1ca12642c9d13f4e931c85bb344eda2b06e47f9fd21775f8"
Sep 06 19:41:22.268838 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:22.268806    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:49.069837 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.069802    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:49.343190 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.341490    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"]
Sep 06 19:41:49.348594 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.347895    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.353246 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.353216    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-413b7f-openshift-default-dockercfg-pjmnp\""
Sep 06 19:41:49.353635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.353307    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"]
Sep 06 19:41:49.358830 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.358802    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:49.412655 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.412613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-istio-token\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.412655 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.412651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1543035a-e232-4651-aaa4-03e777b5cced-istiod-ca-cert\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.412878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.412749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-certs\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.412878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.412790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-envoy\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.412878 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.412807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-data\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.412997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.412880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fslhq\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-kube-api-access-fslhq\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.412997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.412900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-socket\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.412997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.412955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1543035a-e232-4651-aaa4-03e777b5cced-istio-podinfo\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.412997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.412983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-credential-socket\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514380 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-istio-token\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514380 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514276    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1543035a-e232-4651-aaa4-03e777b5cced-istiod-ca-cert\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514380 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-certs\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514380 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-envoy\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-data\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fslhq\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-kube-api-access-fslhq\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-socket\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1543035a-e232-4651-aaa4-03e777b5cced-istio-podinfo\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514519    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-credential-socket\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.514825 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.514805    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-credential-socket\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.515194 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.515153    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-data\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.515240 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.515157    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1543035a-e232-4651-aaa4-03e777b5cced-istiod-ca-cert\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.515275 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.515254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-socket\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.515602 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.515561    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-certs\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.517793 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.517749    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-envoy\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.517971 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.517952    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1543035a-e232-4651-aaa4-03e777b5cced-istio-podinfo\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.521509 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.521481    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-istio-token\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.521700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.521683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fslhq\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-kube-api-access-fslhq\") pod \"e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") " pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.664075 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.664032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:49.722473 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.722439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:49.941826 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.941787    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"]
Sep 06 19:41:49.944339 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:41:49.944307    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1543035a_e232_4651_aaa4_03e777b5cced.slice/crio-18c4cafc224660741c6959a0163d70b8effe401be6c3f5bfab530ba840cd3d38 WatchSource:0}: Error finding container 18c4cafc224660741c6959a0163d70b8effe401be6c3f5bfab530ba840cd3d38: Status 404 returned error can't find the container with id 18c4cafc224660741c6959a0163d70b8effe401be6c3f5bfab530ba840cd3d38
Sep 06 19:41:49.946819 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.946786    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:41:49.946908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.946862    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:41:49.946908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:49.946895    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:41:50.498272 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:50.498231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" event={"ID":"1543035a-e232-4651-aaa4-03e777b5cced","Type":"ContainerStarted","Data":"97f3c3890befcb7560324568f7c351e5a43d067c12e3220bddd278cde28882d1"}
Sep 06 19:41:50.498792 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:50.498291    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" event={"ID":"1543035a-e232-4651-aaa4-03e777b5cced","Type":"ContainerStarted","Data":"18c4cafc224660741c6959a0163d70b8effe401be6c3f5bfab530ba840cd3d38"}
Sep 06 19:41:50.517479 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:50.517418    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" podStartSLOduration=1.5174006960000002 podStartE2EDuration="1.517400696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:41:50.514517952 +0000 UTC m=+1838.794970747" watchObservedRunningTime="2026-09-06 19:41:50.517400696 +0000 UTC m=+1838.797853490"
Sep 06 19:41:50.664363 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:50.664315    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:51.670031 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:51.669998    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:52.406920 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:52.406885    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:52.505180 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:52.505111    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:52.509839 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:52.509814    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:52.510811 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:52.510796    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:41:53.690623 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:53.690590    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:53.791241 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:53.791200    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:53.891966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:53.891915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:54.128725 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:54.128650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:41:54.235236 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:41:54.235202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:12.646666 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:12.646629    2581 generic.go:356] "Generic (PLEG): container finished" podID="1543035a-e232-4651-aaa4-03e777b5cced" containerID="97f3c3890befcb7560324568f7c351e5a43d067c12e3220bddd278cde28882d1" exitCode=0
Sep 06 19:42:12.647138 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:12.646704    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" event={"ID":"1543035a-e232-4651-aaa4-03e777b5cced","Type":"ContainerDied","Data":"97f3c3890befcb7560324568f7c351e5a43d067c12e3220bddd278cde28882d1"}
Sep 06 19:42:12.647239 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:12.647223    2581 scope.go:117] "RemoveContainer" containerID="97f3c3890befcb7560324568f7c351e5a43d067c12e3220bddd278cde28882d1"
Sep 06 19:42:12.648428 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:12.648397    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:42:12.648541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:12.648487    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:42:12.648541 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:12.648536    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 06 19:42:13.654918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:13.654884    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" event={"ID":"1543035a-e232-4651-aaa4-03e777b5cced","Type":"ContainerStarted","Data":"f98f14f01f07d0f5883ec470118366f817db7d4d63ad56db3324d7d654ac1839"}
Sep 06 19:42:13.664343 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:13.664318    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:42:13.664547 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:13.664525    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 06 19:42:13.664596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:13.664572    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 06 19:42:14.664736 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:14.664703    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 06 19:42:14.665183 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:14.664752    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 06 19:42:15.665426 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:15.665390    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 06 19:42:15.665835 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:15.665461    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 06 19:42:16.664842 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:16.664803    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 06 19:42:16.665035 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:16.664862    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 06 19:42:17.373592 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:17.373553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:17.665168 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:17.665128    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 06 19:42:17.665353 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:17.665208    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 06 19:42:18.036077 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.035978    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"]
Sep 06 19:42:18.039872 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.039852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"
Sep 06 19:42:18.042030 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.042006    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:42:18.042196 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.042045    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:42:18.046679 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.046648    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"]
Sep 06 19:42:18.124986 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.124945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da917b9-1155-4496-b9e5-9cf4148cda59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr\" (UID: \"5da917b9-1155-4496-b9e5-9cf4148cda59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"
Sep 06 19:42:18.225747 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.225699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da917b9-1155-4496-b9e5-9cf4148cda59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr\" (UID: \"5da917b9-1155-4496-b9e5-9cf4148cda59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"
Sep 06 19:42:18.226374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.226354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da917b9-1155-4496-b9e5-9cf4148cda59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr\" (UID: \"5da917b9-1155-4496-b9e5-9cf4148cda59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"
Sep 06 19:42:18.352819 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.352726    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"
Sep 06 19:42:18.651846 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.651805    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"]
Sep 06 19:42:18.664789 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.664758    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 06 19:42:18.664918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.664816    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 06 19:42:18.685874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:18.685756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr" event={"ID":"5da917b9-1155-4496-b9e5-9cf4148cda59","Type":"ContainerStarted","Data":"84b6e5a340ceaaf3da08fabace7772ad97238cc685e9f8cabfc9d0f3863c7b2e"}
Sep 06 19:42:19.669188 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:19.669159    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:42:19.669669 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:19.669454    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:42:19.670139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:19.670121    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:42:19.693003 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:19.692969    2581 generic.go:356] "Generic (PLEG): container finished" podID="5da917b9-1155-4496-b9e5-9cf4148cda59" containerID="ddde78e0eabe737f4b0663e422a5882baf560824c075667a6b0a3f7e13680d3d" exitCode=0
Sep 06 19:42:19.693183 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:19.693048    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr" event={"ID":"5da917b9-1155-4496-b9e5-9cf4148cda59","Type":"ContainerDied","Data":"ddde78e0eabe737f4b0663e422a5882baf560824c075667a6b0a3f7e13680d3d"}
Sep 06 19:42:20.837728 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:20.837703    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"
Sep 06 19:42:20.955928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:20.955831    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da917b9-1155-4496-b9e5-9cf4148cda59-maas-api-service-ca\") pod \"5da917b9-1155-4496-b9e5-9cf4148cda59\" (UID: \"5da917b9-1155-4496-b9e5-9cf4148cda59\") "
Sep 06 19:42:20.956238 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:20.956214    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5da917b9-1155-4496-b9e5-9cf4148cda59-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5da917b9-1155-4496-b9e5-9cf4148cda59" (UID: "5da917b9-1155-4496-b9e5-9cf4148cda59"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:42:21.056835 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:21.056790    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da917b9-1155-4496-b9e5-9cf4148cda59-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:21.705319 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:21.705287    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"
Sep 06 19:42:21.705498 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:21.705286    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr" event={"ID":"5da917b9-1155-4496-b9e5-9cf4148cda59","Type":"ContainerDied","Data":"84b6e5a340ceaaf3da08fabace7772ad97238cc685e9f8cabfc9d0f3863c7b2e"}
Sep 06 19:42:21.705498 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:21.705414    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84b6e5a340ceaaf3da08fabace7772ad97238cc685e9f8cabfc9d0f3863c7b2e"
Sep 06 19:42:24.060519 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:24.060479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:24.161106 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:24.161052    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:24.235402 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:24.235337    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"]
Sep 06 19:42:24.237503 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:24.237477    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:24.238471 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:24.238450    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-413b7f-wxxxr"]
Sep 06 19:42:24.266859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:24.266828    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5da917b9-1155-4496-b9e5-9cf4148cda59" path="/var/lib/kubelet/pods/5da917b9-1155-4496-b9e5-9cf4148cda59/volumes"
Sep 06 19:42:24.290216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:24.290185    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:24.589056 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:24.589023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:30.361220 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:30.361184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:30.427540 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:30.427501    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:30.587377 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:30.587325    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"]
Sep 06 19:42:30.587660 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:30.587629    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy" containerID="cri-o://f98f14f01f07d0f5883ec470118366f817db7d4d63ad56db3324d7d654ac1839" gracePeriod=30
Sep 06 19:42:30.609703 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:30.609628    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:34.030734 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.030647    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"]
Sep 06 19:42:34.031136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.031108    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5da917b9-1155-4496-b9e5-9cf4148cda59" containerName="revoke-keys"
Sep 06 19:42:34.031136 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.031121    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5da917b9-1155-4496-b9e5-9cf4148cda59" containerName="revoke-keys"
Sep 06 19:42:34.031226 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.031186    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5da917b9-1155-4496-b9e5-9cf4148cda59" containerName="revoke-keys"
Sep 06 19:42:34.034535 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.034518    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"
Sep 06 19:42:34.036600 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.036565    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:42:34.036731 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.036660    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:42:34.040797 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.040767    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"]
Sep 06 19:42:34.202042 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.202005    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3a39ba-220e-4b6b-97c8-86db09781598-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm\" (UID: \"ca3a39ba-220e-4b6b-97c8-86db09781598\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"
Sep 06 19:42:34.271271 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.271236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:34.303087 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.302975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3a39ba-220e-4b6b-97c8-86db09781598-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm\" (UID: \"ca3a39ba-220e-4b6b-97c8-86db09781598\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"
Sep 06 19:42:34.303659 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.303638    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3a39ba-220e-4b6b-97c8-86db09781598-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm\" (UID: \"ca3a39ba-220e-4b6b-97c8-86db09781598\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"
Sep 06 19:42:34.348738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.348701    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"
Sep 06 19:42:34.624367 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.624341    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"]
Sep 06 19:42:34.626337 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:42:34.626276    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca3a39ba_220e_4b6b_97c8_86db09781598.slice/crio-d5a6e49e20661ae4d9e7c9bc6488c294183ef2915be144e89f88a781b1d4d8c6 WatchSource:0}: Error finding container d5a6e49e20661ae4d9e7c9bc6488c294183ef2915be144e89f88a781b1d4d8c6: Status 404 returned error can't find the container with id d5a6e49e20661ae4d9e7c9bc6488c294183ef2915be144e89f88a781b1d4d8c6
Sep 06 19:42:34.673395 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.673358    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 06 19:42:34.673577 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.673429    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 19:42:34.781139 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:34.781103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm" event={"ID":"ca3a39ba-220e-4b6b-97c8-86db09781598","Type":"ContainerStarted","Data":"d5a6e49e20661ae4d9e7c9bc6488c294183ef2915be144e89f88a781b1d4d8c6"}
Sep 06 19:42:35.789866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:35.789827    2581 generic.go:356] "Generic (PLEG): container finished" podID="ca3a39ba-220e-4b6b-97c8-86db09781598" containerID="1b1c465b8be47f7d19a8d90edf83ac7178582bc1e383dfc2b5bf0df230d5c1d5" exitCode=0
Sep 06 19:42:35.790368 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:35.789911    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm" event={"ID":"ca3a39ba-220e-4b6b-97c8-86db09781598","Type":"ContainerDied","Data":"1b1c465b8be47f7d19a8d90edf83ac7178582bc1e383dfc2b5bf0df230d5c1d5"}
Sep 06 19:42:35.792526 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:35.792501    2581 generic.go:356] "Generic (PLEG): container finished" podID="1543035a-e232-4651-aaa4-03e777b5cced" containerID="f98f14f01f07d0f5883ec470118366f817db7d4d63ad56db3324d7d654ac1839" exitCode=0
Sep 06 19:42:35.792659 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:35.792579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" event={"ID":"1543035a-e232-4651-aaa4-03e777b5cced","Type":"ContainerDied","Data":"f98f14f01f07d0f5883ec470118366f817db7d4d63ad56db3324d7d654ac1839"}
Sep 06 19:42:35.792659 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:35.792636    2581 scope.go:117] "RemoveContainer" containerID="97f3c3890befcb7560324568f7c351e5a43d067c12e3220bddd278cde28882d1"
Sep 06 19:42:35.853032 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:35.853006    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:42:36.020874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.020777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-socket\") pod \"1543035a-e232-4651-aaa4-03e777b5cced\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") "
Sep 06 19:42:36.020874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.020829    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1543035a-e232-4651-aaa4-03e777b5cced-istio-podinfo\") pod \"1543035a-e232-4651-aaa4-03e777b5cced\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") "
Sep 06 19:42:36.020874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.020868    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-envoy\") pod \"1543035a-e232-4651-aaa4-03e777b5cced\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") "
Sep 06 19:42:36.021175 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.020892    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fslhq\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-kube-api-access-fslhq\") pod \"1543035a-e232-4651-aaa4-03e777b5cced\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") "
Sep 06 19:42:36.021175 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.020979    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-certs\") pod \"1543035a-e232-4651-aaa4-03e777b5cced\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") "
Sep 06 19:42:36.021175 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021012    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-credential-socket\") pod \"1543035a-e232-4651-aaa4-03e777b5cced\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") "
Sep 06 19:42:36.021175 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021038    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1543035a-e232-4651-aaa4-03e777b5cced-istiod-ca-cert\") pod \"1543035a-e232-4651-aaa4-03e777b5cced\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") "
Sep 06 19:42:36.021175 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021139    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-data\") pod \"1543035a-e232-4651-aaa4-03e777b5cced\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") "
Sep 06 19:42:36.021175 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021156    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1543035a-e232-4651-aaa4-03e777b5cced" (UID: "1543035a-e232-4651-aaa4-03e777b5cced"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:42:36.021478 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021229    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-istio-token\") pod \"1543035a-e232-4651-aaa4-03e777b5cced\" (UID: \"1543035a-e232-4651-aaa4-03e777b5cced\") "
Sep 06 19:42:36.021478 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021304    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1543035a-e232-4651-aaa4-03e777b5cced" (UID: "1543035a-e232-4651-aaa4-03e777b5cced"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:42:36.021478 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021443    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-data" (OuterVolumeSpecName: "istio-data") pod "1543035a-e232-4651-aaa4-03e777b5cced" (UID: "1543035a-e232-4651-aaa4-03e777b5cced"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:42:36.021644 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021491    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1543035a-e232-4651-aaa4-03e777b5cced-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1543035a-e232-4651-aaa4-03e777b5cced" (UID: "1543035a-e232-4651-aaa4-03e777b5cced"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:42:36.021644 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021519    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1543035a-e232-4651-aaa4-03e777b5cced" (UID: "1543035a-e232-4651-aaa4-03e777b5cced"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:42:36.021718 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021654    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:36.021718 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021667    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:36.021718 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021676    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:36.021718 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021684    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1543035a-e232-4651-aaa4-03e777b5cced-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:36.021718 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.021691    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:36.023326 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.023298    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1543035a-e232-4651-aaa4-03e777b5cced" (UID: "1543035a-e232-4651-aaa4-03e777b5cced"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:42:36.023937 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.023910    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-istio-token" (OuterVolumeSpecName: "istio-token") pod "1543035a-e232-4651-aaa4-03e777b5cced" (UID: "1543035a-e232-4651-aaa4-03e777b5cced"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:42:36.023937 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.023929    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1543035a-e232-4651-aaa4-03e777b5cced-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1543035a-e232-4651-aaa4-03e777b5cced" (UID: "1543035a-e232-4651-aaa4-03e777b5cced"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:42:36.024102 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.023941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-kube-api-access-fslhq" (OuterVolumeSpecName: "kube-api-access-fslhq") pod "1543035a-e232-4651-aaa4-03e777b5cced" (UID: "1543035a-e232-4651-aaa4-03e777b5cced"). InnerVolumeSpecName "kube-api-access-fslhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:42:36.123103 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.123041    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:36.123103 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.123097    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1543035a-e232-4651-aaa4-03e777b5cced-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:36.123103 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.123106    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1543035a-e232-4651-aaa4-03e777b5cced-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:36.123103 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.123118    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fslhq\" (UniqueName: \"kubernetes.io/projected/1543035a-e232-4651-aaa4-03e777b5cced-kube-api-access-fslhq\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:36.799389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.799357    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"
Sep 06 19:42:36.799874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.799406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v" event={"ID":"1543035a-e232-4651-aaa4-03e777b5cced","Type":"ContainerDied","Data":"18c4cafc224660741c6959a0163d70b8effe401be6c3f5bfab530ba840cd3d38"}
Sep 06 19:42:36.799874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.799479    2581 scope.go:117] "RemoveContainer" containerID="f98f14f01f07d0f5883ec470118366f817db7d4d63ad56db3324d7d654ac1839"
Sep 06 19:42:36.818691 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.818658    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"]
Sep 06 19:42:36.822405 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.822367    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-413b7f-openshift-default-647f464c9d-4sk7v"]
Sep 06 19:42:36.939556 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:36.939524    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"
Sep 06 19:42:37.032947 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:37.032903    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3a39ba-220e-4b6b-97c8-86db09781598-maas-api-service-ca\") pod \"ca3a39ba-220e-4b6b-97c8-86db09781598\" (UID: \"ca3a39ba-220e-4b6b-97c8-86db09781598\") "
Sep 06 19:42:37.033314 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:37.033290    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca3a39ba-220e-4b6b-97c8-86db09781598-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca3a39ba-220e-4b6b-97c8-86db09781598" (UID: "ca3a39ba-220e-4b6b-97c8-86db09781598"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:42:37.134147 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:37.134114    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3a39ba-220e-4b6b-97c8-86db09781598-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:42:37.806696 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:37.806666    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"
Sep 06 19:42:37.806696 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:37.806687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm" event={"ID":"ca3a39ba-220e-4b6b-97c8-86db09781598","Type":"ContainerDied","Data":"d5a6e49e20661ae4d9e7c9bc6488c294183ef2915be144e89f88a781b1d4d8c6"}
Sep 06 19:42:37.807213 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:37.806742    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5a6e49e20661ae4d9e7c9bc6488c294183ef2915be144e89f88a781b1d4d8c6"
Sep 06 19:42:38.267222 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:38.267188    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1543035a-e232-4651-aaa4-03e777b5cced" path="/var/lib/kubelet/pods/1543035a-e232-4651-aaa4-03e777b5cced/volumes"
Sep 06 19:42:49.561786 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:49.561749    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:49.626389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:49.626347    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:49.690994 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:49.690955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:49.733339 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:49.733307    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"]
Sep 06 19:42:49.735328 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:49.735303    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-410f4d-p5jbm"]
Sep 06 19:42:49.745004 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:49.744977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:50.108676 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:50.108640    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:42:50.267165 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:50.267133    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca3a39ba-220e-4b6b-97c8-86db09781598" path="/var/lib/kubelet/pods/ca3a39ba-220e-4b6b-97c8-86db09781598/volumes"
Sep 06 19:42:55.139646 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:42:55.139611    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:31.645526 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:31.645475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:31.962044 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:31.961954    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:37.402395 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:37.402355    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:37.458120 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:37.458043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:37.664871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:37.664790    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"]
Sep 06 19:43:37.665022 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:37.665003    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy" containerID="cri-o://31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22" gracePeriod=30
Sep 06 19:43:37.681468 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:37.681441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:38.017282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017193    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"]
Sep 06 19:43:38.017655 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017641    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy"
Sep 06 19:43:38.017698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017657    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy"
Sep 06 19:43:38.017698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017665    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca3a39ba-220e-4b6b-97c8-86db09781598" containerName="revoke-keys"
Sep 06 19:43:38.017698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017671    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca3a39ba-220e-4b6b-97c8-86db09781598" containerName="revoke-keys"
Sep 06 19:43:38.017698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017684    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy"
Sep 06 19:43:38.017698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017689    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy"
Sep 06 19:43:38.017856 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017752    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy"
Sep 06 19:43:38.017856 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017759    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1543035a-e232-4651-aaa4-03e777b5cced" containerName="istio-proxy"
Sep 06 19:43:38.017856 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.017770    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca3a39ba-220e-4b6b-97c8-86db09781598" containerName="revoke-keys"
Sep 06 19:43:38.021690 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.021672    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"
Sep 06 19:43:38.023866 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.023842    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:43:38.023997 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.023847    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:43:38.029077 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.029033    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"]
Sep 06 19:43:38.111225 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.111187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:38.159512 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.159478    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb905b9-2391-45f0-8614-1da0d746b599-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6\" (UID: \"edb905b9-2391-45f0-8614-1da0d746b599\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"
Sep 06 19:43:38.260582 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.260547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb905b9-2391-45f0-8614-1da0d746b599-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6\" (UID: \"edb905b9-2391-45f0-8614-1da0d746b599\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"
Sep 06 19:43:38.261250 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.261224    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb905b9-2391-45f0-8614-1da0d746b599-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6\" (UID: \"edb905b9-2391-45f0-8614-1da0d746b599\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"
Sep 06 19:43:38.333390 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.333293    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"
Sep 06 19:43:38.606587 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:38.606560    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"]
Sep 06 19:43:38.609239 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:43:38.609211    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb905b9_2391_45f0_8614_1da0d746b599.slice/crio-172c731a2dafdd8a45847dfa9db5080471bcf8f5d390887b911a3af338e137c8 WatchSource:0}: Error finding container 172c731a2dafdd8a45847dfa9db5080471bcf8f5d390887b911a3af338e137c8: Status 404 returned error can't find the container with id 172c731a2dafdd8a45847dfa9db5080471bcf8f5d390887b911a3af338e137c8
Sep 06 19:43:39.153761 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:39.153724    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6" event={"ID":"edb905b9-2391-45f0-8614-1da0d746b599","Type":"ContainerStarted","Data":"01babcbaed6e9dfddef7d7c6e7a9b86a04584f6ce2f7cf214bfaeefd4508c696"}
Sep 06 19:43:39.153940 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:39.153778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6" event={"ID":"edb905b9-2391-45f0-8614-1da0d746b599","Type":"ContainerStarted","Data":"172c731a2dafdd8a45847dfa9db5080471bcf8f5d390887b911a3af338e137c8"}
Sep 06 19:43:39.168059 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:39.168004    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6" podStartSLOduration=1.167989119 podStartE2EDuration="1.167989119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:43:39.166346691 +0000 UTC m=+1947.446799482" watchObservedRunningTime="2026-09-06 19:43:39.167989119 +0000 UTC m=+1947.448441945"
Sep 06 19:43:40.164159 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:40.164122    2581 generic.go:356] "Generic (PLEG): container finished" podID="edb905b9-2391-45f0-8614-1da0d746b599" containerID="01babcbaed6e9dfddef7d7c6e7a9b86a04584f6ce2f7cf214bfaeefd4508c696" exitCode=0
Sep 06 19:43:40.164548 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:40.164219    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6" event={"ID":"edb905b9-2391-45f0-8614-1da0d746b599","Type":"ContainerDied","Data":"01babcbaed6e9dfddef7d7c6e7a9b86a04584f6ce2f7cf214bfaeefd4508c696"}
Sep 06 19:43:41.312353 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:41.312327    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"
Sep 06 19:43:41.394245 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:41.394205    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb905b9-2391-45f0-8614-1da0d746b599-maas-api-service-ca\") pod \"edb905b9-2391-45f0-8614-1da0d746b599\" (UID: \"edb905b9-2391-45f0-8614-1da0d746b599\") "
Sep 06 19:43:41.394626 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:41.394601    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edb905b9-2391-45f0-8614-1da0d746b599-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "edb905b9-2391-45f0-8614-1da0d746b599" (UID: "edb905b9-2391-45f0-8614-1da0d746b599"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:43:41.496147 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:41.496028    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb905b9-2391-45f0-8614-1da0d746b599-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:42.176892 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:42.176853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6" event={"ID":"edb905b9-2391-45f0-8614-1da0d746b599","Type":"ContainerDied","Data":"172c731a2dafdd8a45847dfa9db5080471bcf8f5d390887b911a3af338e137c8"}
Sep 06 19:43:42.176892 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:42.176883    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"
Sep 06 19:43:42.177130 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:42.176917    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="172c731a2dafdd8a45847dfa9db5080471bcf8f5d390887b911a3af338e137c8"
Sep 06 19:43:42.951364 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:42.951336    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:43:43.112998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.112962    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-credential-socket\") pod \"58e676d4-0a47-4604-a4be-6f3158cce967\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") "
Sep 06 19:43:43.112998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113003    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbmj4\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-kube-api-access-cbmj4\") pod \"58e676d4-0a47-4604-a4be-6f3158cce967\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") "
Sep 06 19:43:43.113258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113042    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58e676d4-0a47-4604-a4be-6f3158cce967-istio-podinfo\") pod \"58e676d4-0a47-4604-a4be-6f3158cce967\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") "
Sep 06 19:43:43.113258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113104    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-istio-token\") pod \"58e676d4-0a47-4604-a4be-6f3158cce967\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") "
Sep 06 19:43:43.113258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113166    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-certs\") pod \"58e676d4-0a47-4604-a4be-6f3158cce967\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") "
Sep 06 19:43:43.113258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113193    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-envoy\") pod \"58e676d4-0a47-4604-a4be-6f3158cce967\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") "
Sep 06 19:43:43.113258 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113215    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-data\") pod \"58e676d4-0a47-4604-a4be-6f3158cce967\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") "
Sep 06 19:43:43.113515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113269    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58e676d4-0a47-4604-a4be-6f3158cce967-istiod-ca-cert\") pod \"58e676d4-0a47-4604-a4be-6f3158cce967\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") "
Sep 06 19:43:43.113515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113291    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-socket\") pod \"58e676d4-0a47-4604-a4be-6f3158cce967\" (UID: \"58e676d4-0a47-4604-a4be-6f3158cce967\") "
Sep 06 19:43:43.113515 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113333    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "58e676d4-0a47-4604-a4be-6f3158cce967" (UID: "58e676d4-0a47-4604-a4be-6f3158cce967"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:43:43.113691 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113521    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "58e676d4-0a47-4604-a4be-6f3158cce967" (UID: "58e676d4-0a47-4604-a4be-6f3158cce967"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:43:43.113778 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113683    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58e676d4-0a47-4604-a4be-6f3158cce967-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "58e676d4-0a47-4604-a4be-6f3158cce967" (UID: "58e676d4-0a47-4604-a4be-6f3158cce967"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:43:43.113970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113758    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:43.113970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113783    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-data" (OuterVolumeSpecName: "istio-data") pod "58e676d4-0a47-4604-a4be-6f3158cce967" (UID: "58e676d4-0a47-4604-a4be-6f3158cce967"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:43:43.113970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113808    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:43.113970 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.113812    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "58e676d4-0a47-4604-a4be-6f3158cce967" (UID: "58e676d4-0a47-4604-a4be-6f3158cce967"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:43:43.115511 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.115484    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-istio-token" (OuterVolumeSpecName: "istio-token") pod "58e676d4-0a47-4604-a4be-6f3158cce967" (UID: "58e676d4-0a47-4604-a4be-6f3158cce967"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:43:43.115612 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.115517    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-kube-api-access-cbmj4" (OuterVolumeSpecName: "kube-api-access-cbmj4") pod "58e676d4-0a47-4604-a4be-6f3158cce967" (UID: "58e676d4-0a47-4604-a4be-6f3158cce967"). InnerVolumeSpecName "kube-api-access-cbmj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:43:43.115612 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.115547    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58e676d4-0a47-4604-a4be-6f3158cce967-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "58e676d4-0a47-4604-a4be-6f3158cce967" (UID: "58e676d4-0a47-4604-a4be-6f3158cce967"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:43:43.115687 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.115638    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "58e676d4-0a47-4604-a4be-6f3158cce967" (UID: "58e676d4-0a47-4604-a4be-6f3158cce967"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:43:43.185287 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.185187    2581 generic.go:356] "Generic (PLEG): container finished" podID="58e676d4-0a47-4604-a4be-6f3158cce967" containerID="31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22" exitCode=0
Sep 06 19:43:43.185287 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.185230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" event={"ID":"58e676d4-0a47-4604-a4be-6f3158cce967","Type":"ContainerDied","Data":"31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22"}
Sep 06 19:43:43.185287 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.185242    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"
Sep 06 19:43:43.185287 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.185283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc" event={"ID":"58e676d4-0a47-4604-a4be-6f3158cce967","Type":"ContainerDied","Data":"d668b9d002b5501210fdf23a4f81a9a34d5cd99ad51333f8deeb0b64f19f785e"}
Sep 06 19:43:43.185591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.185312    2581 scope.go:117] "RemoveContainer" containerID="31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22"
Sep 06 19:43:43.197384 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.197361    2581 scope.go:117] "RemoveContainer" containerID="00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3"
Sep 06 19:43:43.206987 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.206675    2581 scope.go:117] "RemoveContainer" containerID="31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22"
Sep 06 19:43:43.207285 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:43:43.207135    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22\": container with ID starting with 31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22 not found: ID does not exist" containerID="31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22"
Sep 06 19:43:43.207285 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.207172    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22"} err="failed to get container status \"31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22\": rpc error: code = NotFound desc = could not find container \"31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22\": container with ID starting with 31d371c2949b53ce8b2dbd6d70e32132d60037c892df229e89b27c9bf98b6b22 not found: ID does not exist"
Sep 06 19:43:43.207285 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.207193    2581 scope.go:117] "RemoveContainer" containerID="00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3"
Sep 06 19:43:43.207540 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:43:43.207511    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3\": container with ID starting with 00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3 not found: ID does not exist" containerID="00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3"
Sep 06 19:43:43.207540 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.207530    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3"} err="failed to get container status \"00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3\": rpc error: code = NotFound desc = could not find container \"00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3\": container with ID starting with 00f8c0901177b68b35d21d549feef3396da5e072fcabbf00ea2ecc1de88b7ed3 not found: ID does not exist"
Sep 06 19:43:43.209340 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.209317    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"]
Sep 06 19:43:43.214573 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.214543    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-410f4d-openshift-default-5bf55d65c6-d5nfc"]
Sep 06 19:43:43.214672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.214597    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:43.214672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.214618    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:43.214672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.214626    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58e676d4-0a47-4604-a4be-6f3158cce967-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:43.214672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.214635    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58e676d4-0a47-4604-a4be-6f3158cce967-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:43.214672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.214645    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbmj4\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-kube-api-access-cbmj4\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:43.214672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.214653    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58e676d4-0a47-4604-a4be-6f3158cce967-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:43.214672 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:43.214661    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58e676d4-0a47-4604-a4be-6f3158cce967-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:44.266337 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:44.266301    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" path="/var/lib/kubelet/pods/58e676d4-0a47-4604-a4be-6f3158cce967/volumes"
Sep 06 19:43:48.459852 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:48.459813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:48.560132 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:48.560095    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:48.626189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:48.626142    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"]
Sep 06 19:43:48.628315 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:48.628286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:48.628579 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:48.628559    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-88ac22-m89l6"]
Sep 06 19:43:48.686596 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:48.686556    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:49.016765 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:49.016731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:50.266714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:50.266680    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edb905b9-2391-45f0-8614-1da0d746b599" path="/var/lib/kubelet/pods/edb905b9-2391-45f0-8614-1da0d746b599/volumes"
Sep 06 19:43:54.759289 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:54.759253    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:54.816498 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:54.816459    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:54.999587 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:54.999550    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"]
Sep 06 19:43:54.999800 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:54.999779    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy" containerID="cri-o://ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d" gracePeriod=30
Sep 06 19:43:55.020205 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.020114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:55.257717 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.257679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:43:55.351730 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.351647    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"]
Sep 06 19:43:55.352090 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.352077    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy"
Sep 06 19:43:55.352148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.352091    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy"
Sep 06 19:43:55.352148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.352101    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edb905b9-2391-45f0-8614-1da0d746b599" containerName="revoke-keys"
Sep 06 19:43:55.352148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.352106    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="edb905b9-2391-45f0-8614-1da0d746b599" containerName="revoke-keys"
Sep 06 19:43:55.352148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.352118    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy"
Sep 06 19:43:55.352148 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.352123    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy"
Sep 06 19:43:55.352302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.352195    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="edb905b9-2391-45f0-8614-1da0d746b599" containerName="revoke-keys"
Sep 06 19:43:55.352302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.352202    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy"
Sep 06 19:43:55.352302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.352211    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="58e676d4-0a47-4604-a4be-6f3158cce967" containerName="istio-proxy"
Sep 06 19:43:55.355651 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.355635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"
Sep 06 19:43:55.357663 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.357638    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:43:55.357788 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.357684    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:43:55.361094 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.361039    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"]
Sep 06 19:43:55.436593 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.436562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad505aab-7f28-46a8-998a-2f513b5cf348-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm\" (UID: \"ad505aab-7f28-46a8-998a-2f513b5cf348\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"
Sep 06 19:43:55.537183 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.537145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad505aab-7f28-46a8-998a-2f513b5cf348-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm\" (UID: \"ad505aab-7f28-46a8-998a-2f513b5cf348\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"
Sep 06 19:43:55.537874 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.537850    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad505aab-7f28-46a8-998a-2f513b5cf348-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm\" (UID: \"ad505aab-7f28-46a8-998a-2f513b5cf348\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"
Sep 06 19:43:55.668766 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.668731    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"
Sep 06 19:43:55.939212 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:55.939170    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"]
Sep 06 19:43:55.941373 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:43:55.941342    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad505aab_7f28_46a8_998a_2f513b5cf348.slice/crio-51ca8db857e49897454e04815ea035c825a2eab1a217d22f139c965be7ac230c WatchSource:0}: Error finding container 51ca8db857e49897454e04815ea035c825a2eab1a217d22f139c965be7ac230c: Status 404 returned error can't find the container with id 51ca8db857e49897454e04815ea035c825a2eab1a217d22f139c965be7ac230c
Sep 06 19:43:56.265808 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:56.265704    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm" event={"ID":"ad505aab-7f28-46a8-998a-2f513b5cf348","Type":"ContainerStarted","Data":"db443c872713ad808d06523ffc927931c987b8f9f25547161924facc6cae89b3"}
Sep 06 19:43:56.265808 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:56.265758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm" event={"ID":"ad505aab-7f28-46a8-998a-2f513b5cf348","Type":"ContainerStarted","Data":"51ca8db857e49897454e04815ea035c825a2eab1a217d22f139c965be7ac230c"}
Sep 06 19:43:56.279918 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:56.279866    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm" podStartSLOduration=1.279846324 podStartE2EDuration="1.279846324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:43:56.27798383 +0000 UTC m=+1964.558436625" watchObservedRunningTime="2026-09-06 19:43:56.279846324 +0000 UTC m=+1964.560299118"
Sep 06 19:43:57.272292 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:57.272255    2581 generic.go:356] "Generic (PLEG): container finished" podID="ad505aab-7f28-46a8-998a-2f513b5cf348" containerID="db443c872713ad808d06523ffc927931c987b8f9f25547161924facc6cae89b3" exitCode=0
Sep 06 19:43:57.272714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:57.272333    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm" event={"ID":"ad505aab-7f28-46a8-998a-2f513b5cf348","Type":"ContainerDied","Data":"db443c872713ad808d06523ffc927931c987b8f9f25547161924facc6cae89b3"}
Sep 06 19:43:58.409355 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:58.409330    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"
Sep 06 19:43:58.465203 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:58.465169    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad505aab-7f28-46a8-998a-2f513b5cf348-maas-api-service-ca\") pod \"ad505aab-7f28-46a8-998a-2f513b5cf348\" (UID: \"ad505aab-7f28-46a8-998a-2f513b5cf348\") "
Sep 06 19:43:58.465530 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:58.465506    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad505aab-7f28-46a8-998a-2f513b5cf348-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ad505aab-7f28-46a8-998a-2f513b5cf348" (UID: "ad505aab-7f28-46a8-998a-2f513b5cf348"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:43:58.465613 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:58.465602    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad505aab-7f28-46a8-998a-2f513b5cf348-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:43:59.284282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:59.284242    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"
Sep 06 19:43:59.284282 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:59.284276    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm" event={"ID":"ad505aab-7f28-46a8-998a-2f513b5cf348","Type":"ContainerDied","Data":"51ca8db857e49897454e04815ea035c825a2eab1a217d22f139c965be7ac230c"}
Sep 06 19:43:59.284489 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:43:59.284322    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51ca8db857e49897454e04815ea035c825a2eab1a217d22f139c965be7ac230c"
Sep 06 19:44:00.268189 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.268125    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:44:00.293718 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.293680    2581 generic.go:356] "Generic (PLEG): container finished" podID="3bf90734-d656-45f6-a506-2928819dd05b" containerID="ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d" exitCode=0
Sep 06 19:44:00.293900 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.293728    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"
Sep 06 19:44:00.293900 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.293763    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" event={"ID":"3bf90734-d656-45f6-a506-2928819dd05b","Type":"ContainerDied","Data":"ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d"}
Sep 06 19:44:00.293900 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.293820    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7" event={"ID":"3bf90734-d656-45f6-a506-2928819dd05b","Type":"ContainerDied","Data":"b9cea15b8288d104e5fea7b12c8722784fc0493c5b6ccb478767138949fd81c1"}
Sep 06 19:44:00.293900 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.293849    2581 scope.go:117] "RemoveContainer" containerID="ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d"
Sep 06 19:44:00.303327 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.303305    2581 scope.go:117] "RemoveContainer" containerID="0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f"
Sep 06 19:44:00.312143 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.312120    2581 scope.go:117] "RemoveContainer" containerID="ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d"
Sep 06 19:44:00.312487 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:44:00.312460    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d\": container with ID starting with ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d not found: ID does not exist" containerID="ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d"
Sep 06 19:44:00.312553 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.312499    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d"} err="failed to get container status \"ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d\": rpc error: code = NotFound desc = could not find container \"ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d\": container with ID starting with ba84f0909587150d63ad22c3c9a160e1b4879d778e3424c8b1563a12eb77e27d not found: ID does not exist"
Sep 06 19:44:00.312553 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.312520    2581 scope.go:117] "RemoveContainer" containerID="0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f"
Sep 06 19:44:00.312810 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:44:00.312793    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f\": container with ID starting with 0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f not found: ID does not exist" containerID="0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f"
Sep 06 19:44:00.312859 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.312825    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f"} err="failed to get container status \"0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f\": rpc error: code = NotFound desc = could not find container \"0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f\": container with ID starting with 0ca45734f1e2a36ba96ef168676f3fcff93cd36b7ac98772e92f1ae3accbef4f not found: ID does not exist"
Sep 06 19:44:00.383698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.383664    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-istio-token\") pod \"3bf90734-d656-45f6-a506-2928819dd05b\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") "
Sep 06 19:44:00.383698 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.383704    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-certs\") pod \"3bf90734-d656-45f6-a506-2928819dd05b\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") "
Sep 06 19:44:00.383945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.383744    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-envoy\") pod \"3bf90734-d656-45f6-a506-2928819dd05b\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") "
Sep 06 19:44:00.383945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.383786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf90734-d656-45f6-a506-2928819dd05b-istiod-ca-cert\") pod \"3bf90734-d656-45f6-a506-2928819dd05b\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") "
Sep 06 19:44:00.383945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.383811    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-data\") pod \"3bf90734-d656-45f6-a506-2928819dd05b\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") "
Sep 06 19:44:00.383945 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.383834    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfjg7\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-kube-api-access-lfjg7\") pod \"3bf90734-d656-45f6-a506-2928819dd05b\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") "
Sep 06 19:44:00.384243 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.383946    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-socket\") pod \"3bf90734-d656-45f6-a506-2928819dd05b\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") "
Sep 06 19:44:00.384243 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.383985    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-credential-socket\") pod \"3bf90734-d656-45f6-a506-2928819dd05b\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") "
Sep 06 19:44:00.384243 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384022    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf90734-d656-45f6-a506-2928819dd05b-istio-podinfo\") pod \"3bf90734-d656-45f6-a506-2928819dd05b\" (UID: \"3bf90734-d656-45f6-a506-2928819dd05b\") "
Sep 06 19:44:00.384243 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384086    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3bf90734-d656-45f6-a506-2928819dd05b" (UID: "3bf90734-d656-45f6-a506-2928819dd05b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:00.384243 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384195    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3bf90734-d656-45f6-a506-2928819dd05b" (UID: "3bf90734-d656-45f6-a506-2928819dd05b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:00.384479 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384255    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bf90734-d656-45f6-a506-2928819dd05b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3bf90734-d656-45f6-a506-2928819dd05b" (UID: "3bf90734-d656-45f6-a506-2928819dd05b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:44:00.384479 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384301    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3bf90734-d656-45f6-a506-2928819dd05b" (UID: "3bf90734-d656-45f6-a506-2928819dd05b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:00.384591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384480    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:00.384591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384498    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:00.384591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384511    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf90734-d656-45f6-a506-2928819dd05b-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:00.384591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384511    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3bf90734-d656-45f6-a506-2928819dd05b" (UID: "3bf90734-d656-45f6-a506-2928819dd05b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:00.384591 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.384524    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:00.386209 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.386181    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3bf90734-d656-45f6-a506-2928819dd05b" (UID: "3bf90734-d656-45f6-a506-2928819dd05b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:44:00.386355 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.386332    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-kube-api-access-lfjg7" (OuterVolumeSpecName: "kube-api-access-lfjg7") pod "3bf90734-d656-45f6-a506-2928819dd05b" (UID: "3bf90734-d656-45f6-a506-2928819dd05b"). InnerVolumeSpecName "kube-api-access-lfjg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:44:00.386653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.386635    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3bf90734-d656-45f6-a506-2928819dd05b" (UID: "3bf90734-d656-45f6-a506-2928819dd05b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:00.386653 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.386642    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3bf90734-d656-45f6-a506-2928819dd05b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3bf90734-d656-45f6-a506-2928819dd05b" (UID: "3bf90734-d656-45f6-a506-2928819dd05b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:44:00.485919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.485880    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:00.485919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.485916    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf90734-d656-45f6-a506-2928819dd05b-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:00.485919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.485928    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:00.486215 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.485939    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf90734-d656-45f6-a506-2928819dd05b-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:00.486215 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.485950    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfjg7\" (UniqueName: \"kubernetes.io/projected/3bf90734-d656-45f6-a506-2928819dd05b-kube-api-access-lfjg7\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:00.614981 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.614947    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"]
Sep 06 19:44:00.616935 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:00.616912    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-88ac22-openshift-default-6d67f67bb4-c2mj7"]
Sep 06 19:44:02.266950 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:02.266915    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bf90734-d656-45f6-a506-2928819dd05b" path="/var/lib/kubelet/pods/3bf90734-d656-45f6-a506-2928819dd05b/volumes"
Sep 06 19:44:02.505214 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:02.505179    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:02.556008 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:02.555922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:05.856592 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:05.856496    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:05.936188 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:05.936147    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:05.996216 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:05.996179    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:06.096007 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:06.095972    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:06.120057 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:06.119953    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"]
Sep 06 19:44:06.121341 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:06.121317    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-e5ef25-g5hnm"]
Sep 06 19:44:06.266419 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:06.266373    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad505aab-7f28-46a8-998a-2f513b5cf348" path="/var/lib/kubelet/pods/ad505aab-7f28-46a8-998a-2f513b5cf348/volumes"
Sep 06 19:44:06.488500 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:06.488467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:12.374194 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.374155    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:12.478078 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.478029    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:12.609930 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.609892    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"]
Sep 06 19:44:12.610210 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.610172    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy" containerID="cri-o://ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10" gracePeriod=30
Sep 06 19:44:12.639397 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.639366    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:12.723046 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.723002    2581 scope.go:117] "RemoveContainer" containerID="e134ffc7f9dffb2c04728ad0467fc5a27c04b66071e67d6dd110e622b8267ae6"
Sep 06 19:44:12.963382 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963294    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"]
Sep 06 19:44:12.963712 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963700    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy"
Sep 06 19:44:12.963758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963714    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy"
Sep 06 19:44:12.963758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963728    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad505aab-7f28-46a8-998a-2f513b5cf348" containerName="revoke-keys"
Sep 06 19:44:12.963758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963734    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad505aab-7f28-46a8-998a-2f513b5cf348" containerName="revoke-keys"
Sep 06 19:44:12.963758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963744    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy"
Sep 06 19:44:12.963758 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963749    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy"
Sep 06 19:44:12.963909 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963820    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad505aab-7f28-46a8-998a-2f513b5cf348" containerName="revoke-keys"
Sep 06 19:44:12.963909 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963828    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy"
Sep 06 19:44:12.963909 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.963837    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bf90734-d656-45f6-a506-2928819dd05b" containerName="istio-proxy"
Sep 06 19:44:12.968529 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.968510    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"
Sep 06 19:44:12.970980 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.970951    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:44:12.971291 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.971272    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 19:44:12.981117 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:12.978647    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"]
Sep 06 19:44:13.049217 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:13.049174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:13.111301 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:13.111267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0bef290-31f5-499f-9f13-6e0ecff9f764-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz\" (UID: \"d0bef290-31f5-499f-9f13-6e0ecff9f764\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"
Sep 06 19:44:13.212327 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:13.212287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0bef290-31f5-499f-9f13-6e0ecff9f764-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz\" (UID: \"d0bef290-31f5-499f-9f13-6e0ecff9f764\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"
Sep 06 19:44:13.212990 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:13.212966    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0bef290-31f5-499f-9f13-6e0ecff9f764-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz\" (UID: \"d0bef290-31f5-499f-9f13-6e0ecff9f764\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"
Sep 06 19:44:13.285316 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:13.285211    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"
Sep 06 19:44:13.559364 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:13.559332    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"]
Sep 06 19:44:13.561406 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:44:13.561380    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0bef290_31f5_499f_9f13_6e0ecff9f764.slice/crio-bbdc2d1c69148087ffcffbf42e69c18a9c8c1ca79bd20ee371efcf447fff6255 WatchSource:0}: Error finding container bbdc2d1c69148087ffcffbf42e69c18a9c8c1ca79bd20ee371efcf447fff6255: Status 404 returned error can't find the container with id bbdc2d1c69148087ffcffbf42e69c18a9c8c1ca79bd20ee371efcf447fff6255
Sep 06 19:44:14.386177 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:14.386136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz" event={"ID":"d0bef290-31f5-499f-9f13-6e0ecff9f764","Type":"ContainerStarted","Data":"e1844eab4038b422dd8f3653bf950b73a8027b9a1a95ebce092d4a058aebab6a"}
Sep 06 19:44:14.386407 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:14.386205    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz" event={"ID":"d0bef290-31f5-499f-9f13-6e0ecff9f764","Type":"ContainerStarted","Data":"bbdc2d1c69148087ffcffbf42e69c18a9c8c1ca79bd20ee371efcf447fff6255"}
Sep 06 19:44:14.401919 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:14.401874    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz" podStartSLOduration=2.401856262 podStartE2EDuration="2.401856262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:44:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:44:14.400179429 +0000 UTC m=+1982.680632217" watchObservedRunningTime="2026-09-06 19:44:14.401856262 +0000 UTC m=+1982.682309082"
Sep 06 19:44:15.394020 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:15.393988    2581 generic.go:356] "Generic (PLEG): container finished" podID="d0bef290-31f5-499f-9f13-6e0ecff9f764" containerID="e1844eab4038b422dd8f3653bf950b73a8027b9a1a95ebce092d4a058aebab6a" exitCode=0
Sep 06 19:44:15.394463 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:15.394100    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz" event={"ID":"d0bef290-31f5-499f-9f13-6e0ecff9f764","Type":"ContainerDied","Data":"e1844eab4038b422dd8f3653bf950b73a8027b9a1a95ebce092d4a058aebab6a"}
Sep 06 19:44:16.543996 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:16.543968    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"
Sep 06 19:44:16.647879 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:16.647833    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0bef290-31f5-499f-9f13-6e0ecff9f764-maas-api-service-ca\") pod \"d0bef290-31f5-499f-9f13-6e0ecff9f764\" (UID: \"d0bef290-31f5-499f-9f13-6e0ecff9f764\") "
Sep 06 19:44:16.648266 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:16.648242    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0bef290-31f5-499f-9f13-6e0ecff9f764-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d0bef290-31f5-499f-9f13-6e0ecff9f764" (UID: "d0bef290-31f5-499f-9f13-6e0ecff9f764"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:44:16.748889 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:16.748785    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0bef290-31f5-499f-9f13-6e0ecff9f764-maas-api-service-ca\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:17.406399 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.406361    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"
Sep 06 19:44:17.406623 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.406361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz" event={"ID":"d0bef290-31f5-499f-9f13-6e0ecff9f764","Type":"ContainerDied","Data":"bbdc2d1c69148087ffcffbf42e69c18a9c8c1ca79bd20ee371efcf447fff6255"}
Sep 06 19:44:17.406623 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.406497    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbdc2d1c69148087ffcffbf42e69c18a9c8c1ca79bd20ee371efcf447fff6255"
Sep 06 19:44:17.470900 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.470856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:17.864413 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.864388    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:44:17.961281 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961240    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-data\") pod \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") "
Sep 06 19:44:17.961479 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961307    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istiod-ca-cert\") pod \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") "
Sep 06 19:44:17.961479 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961347    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-certs\") pod \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") "
Sep 06 19:44:17.961479 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961384    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-envoy\") pod \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") "
Sep 06 19:44:17.961479 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961409    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-credential-socket\") pod \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") "
Sep 06 19:44:17.961479 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961444    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-token\") pod \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") "
Sep 06 19:44:17.961739 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961489    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zxvj\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-kube-api-access-8zxvj\") pod \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") "
Sep 06 19:44:17.961739 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-socket\") pod \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") "
Sep 06 19:44:17.961739 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961555    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-podinfo\") pod \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\" (UID: \"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42\") "
Sep 06 19:44:17.961739 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961556    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" (UID: "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:17.961739 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961643    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" (UID: "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:17.961979 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961805    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" (UID: "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:44:17.961979 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.961947    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" (UID: "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:17.962117 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.962021    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:17.962117 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.962041    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:17.962117 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.962054    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:17.962229 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.962020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" (UID: "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:17.963783 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.963755    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" (UID: "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:17.964021 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.964001    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-kube-api-access-8zxvj" (OuterVolumeSpecName: "kube-api-access-8zxvj") pod "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" (UID: "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42"). InnerVolumeSpecName "kube-api-access-8zxvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:44:17.964426 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.964407    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" (UID: "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:44:17.964426 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:17.964410    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" (UID: "ae671dd5-c6ae-4bc1-8587-1f4906ec3f42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:44:18.063253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.063143    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zxvj\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-kube-api-access-8zxvj\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:18.063253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.063181    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:18.063253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.063195    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:18.063253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.063208    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:18.063253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.063220    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:18.063253 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.063230    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:18.414114 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.414048    2581 generic.go:356] "Generic (PLEG): container finished" podID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerID="ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10" exitCode=0
Sep 06 19:44:18.414281 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.414129    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"
Sep 06 19:44:18.414281 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.414140    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" event={"ID":"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42","Type":"ContainerDied","Data":"ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10"}
Sep 06 19:44:18.414281 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.414204    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p" event={"ID":"ae671dd5-c6ae-4bc1-8587-1f4906ec3f42","Type":"ContainerDied","Data":"066a72c00be288969abbab7759e2cd50c344da3ccbc274328bb456d2b6f46189"}
Sep 06 19:44:18.414281 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.414236    2581 scope.go:117] "RemoveContainer" containerID="ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10"
Sep 06 19:44:18.428817 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.428791    2581 scope.go:117] "RemoveContainer" containerID="3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3"
Sep 06 19:44:18.436835 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.436801    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"]
Sep 06 19:44:18.439962 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.439936    2581 scope.go:117] "RemoveContainer" containerID="ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10"
Sep 06 19:44:18.440403 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:44:18.440376    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10\": container with ID starting with ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10 not found: ID does not exist" containerID="ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10"
Sep 06 19:44:18.440461 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.440413    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10"} err="failed to get container status \"ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10\": rpc error: code = NotFound desc = could not find container \"ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10\": container with ID starting with ceb77e8a5c35b4152a1b3e1b7d91db3789cce7b4b40cb214ab05615837377d10 not found: ID does not exist"
Sep 06 19:44:18.440461 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.440433    2581 scope.go:117] "RemoveContainer" containerID="3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3"
Sep 06 19:44:18.440757 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:44:18.440735    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3\": container with ID starting with 3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3 not found: ID does not exist" containerID="3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3"
Sep 06 19:44:18.440807 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.440765    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3"} err="failed to get container status \"3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3\": rpc error: code = NotFound desc = could not find container \"3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3\": container with ID starting with 3bf50def6a8776f56288c1f3c22cccdac373c1b713aca51be9467310ae93e8a3 not found: ID does not exist"
Sep 06 19:44:18.440843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:18.440822    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5ef25-openshift-default-7959cdd7d4-hhw5p"]
Sep 06 19:44:20.266553 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:20.266518    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" path="/var/lib/kubelet/pods/ae671dd5-c6ae-4bc1-8587-1f4906ec3f42/volumes"
Sep 06 19:44:23.449266 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:23.449223    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:23.509420 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:23.509385    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:23.609944 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:23.609913    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:23.617840 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:23.617802    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"]
Sep 06 19:44:23.623701 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:23.623672    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-476f98-zxkxz"]
Sep 06 19:44:23.695435 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:23.695402    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:23.981381 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:23.981332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:24.270903 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:24.270816    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0bef290-31f5-499f-9f13-6e0ecff9f764" path="/var/lib/kubelet/pods/d0bef290-31f5-499f-9f13-6e0ecff9f764/volumes"
Sep 06 19:44:29.620898 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:29.620862    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:29.721312 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:29.721281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:29.853507 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:29.853469    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"]
Sep 06 19:44:29.853735 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:29.853712    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy" containerID="cri-o://8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9" gracePeriod=30
Sep 06 19:44:29.870305 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:29.870270    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:44:35.135456 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.135432    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:44:35.230114 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230046    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-credential-socket\") pod \"83122217-d60d-4ef3-9762-8d24d44641d8\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") "
Sep 06 19:44:35.230114 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230119    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83122217-d60d-4ef3-9762-8d24d44641d8-istio-podinfo\") pod \"83122217-d60d-4ef3-9762-8d24d44641d8\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") "
Sep 06 19:44:35.230374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230154    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-data\") pod \"83122217-d60d-4ef3-9762-8d24d44641d8\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") "
Sep 06 19:44:35.230374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230215    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxqx4\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-kube-api-access-dxqx4\") pod \"83122217-d60d-4ef3-9762-8d24d44641d8\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") "
Sep 06 19:44:35.230374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230241    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83122217-d60d-4ef3-9762-8d24d44641d8-istiod-ca-cert\") pod \"83122217-d60d-4ef3-9762-8d24d44641d8\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") "
Sep 06 19:44:35.230374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230261    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-istio-token\") pod \"83122217-d60d-4ef3-9762-8d24d44641d8\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") "
Sep 06 19:44:35.230374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230326    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-certs\") pod \"83122217-d60d-4ef3-9762-8d24d44641d8\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") "
Sep 06 19:44:35.230374 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230368    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-socket\") pod \"83122217-d60d-4ef3-9762-8d24d44641d8\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") "
Sep 06 19:44:35.230681 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230371    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83122217-d60d-4ef3-9762-8d24d44641d8" (UID: "83122217-d60d-4ef3-9762-8d24d44641d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:35.230681 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230407    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-envoy\") pod \"83122217-d60d-4ef3-9762-8d24d44641d8\" (UID: \"83122217-d60d-4ef3-9762-8d24d44641d8\") "
Sep 06 19:44:35.230681 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230574    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "83122217-d60d-4ef3-9762-8d24d44641d8" (UID: "83122217-d60d-4ef3-9762-8d24d44641d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:35.230865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230683    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83122217-d60d-4ef3-9762-8d24d44641d8" (UID: "83122217-d60d-4ef3-9762-8d24d44641d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:35.230865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230797    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83122217-d60d-4ef3-9762-8d24d44641d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83122217-d60d-4ef3-9762-8d24d44641d8" (UID: "83122217-d60d-4ef3-9762-8d24d44641d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 19:44:35.230865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230800    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83122217-d60d-4ef3-9762-8d24d44641d8" (UID: "83122217-d60d-4ef3-9762-8d24d44641d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:35.230865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230833    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-credential-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:35.230865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230855    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-data\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:35.230865 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.230869    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-certs\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:35.232530 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.232504    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83122217-d60d-4ef3-9762-8d24d44641d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83122217-d60d-4ef3-9762-8d24d44641d8" (UID: "83122217-d60d-4ef3-9762-8d24d44641d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 19:44:35.232691 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.232674    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-kube-api-access-dxqx4" (OuterVolumeSpecName: "kube-api-access-dxqx4") pod "83122217-d60d-4ef3-9762-8d24d44641d8" (UID: "83122217-d60d-4ef3-9762-8d24d44641d8"). InnerVolumeSpecName "kube-api-access-dxqx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:44:35.232744 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.232704    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83122217-d60d-4ef3-9762-8d24d44641d8" (UID: "83122217-d60d-4ef3-9762-8d24d44641d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 19:44:35.232804 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.232785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "83122217-d60d-4ef3-9762-8d24d44641d8" (UID: "83122217-d60d-4ef3-9762-8d24d44641d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 19:44:35.332457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.332415    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxqx4\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-kube-api-access-dxqx4\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:35.332457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.332447    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83122217-d60d-4ef3-9762-8d24d44641d8-istiod-ca-cert\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:35.332457 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.332459    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83122217-d60d-4ef3-9762-8d24d44641d8-istio-token\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:35.332751 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.332474    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-workload-socket\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:35.332751 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.332488    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83122217-d60d-4ef3-9762-8d24d44641d8-istio-envoy\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:35.332751 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.332501    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83122217-d60d-4ef3-9762-8d24d44641d8-istio-podinfo\") on node \"ip-10-0-135-200.ec2.internal\" DevicePath \"\""
Sep 06 19:44:35.521173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.521054    2581 generic.go:356] "Generic (PLEG): container finished" podID="83122217-d60d-4ef3-9762-8d24d44641d8" containerID="8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9" exitCode=0
Sep 06 19:44:35.521173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.521110    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" event={"ID":"83122217-d60d-4ef3-9762-8d24d44641d8","Type":"ContainerDied","Data":"8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9"}
Sep 06 19:44:35.521173 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.521130    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"
Sep 06 19:44:35.521439 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.521176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx" event={"ID":"83122217-d60d-4ef3-9762-8d24d44641d8","Type":"ContainerDied","Data":"3d09f726f5b77c3ecdff6731a3d3eeb360947c0595372372875418433b210f85"}
Sep 06 19:44:35.521439 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.521206    2581 scope.go:117] "RemoveContainer" containerID="8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9"
Sep 06 19:44:35.531697 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.531676    2581 scope.go:117] "RemoveContainer" containerID="d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa"
Sep 06 19:44:35.540372 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.540351    2581 scope.go:117] "RemoveContainer" containerID="8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9"
Sep 06 19:44:35.540665 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:44:35.540644    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9\": container with ID starting with 8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9 not found: ID does not exist" containerID="8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9"
Sep 06 19:44:35.540724 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.540674    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9"} err="failed to get container status \"8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9\": rpc error: code = NotFound desc = could not find container \"8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9\": container with ID starting with 8a828e5eac8bb3a3d316bfa9fa9b2ec7d3c2129bce891b9d3e9812fb22d356e9 not found: ID does not exist"
Sep 06 19:44:35.540724 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.540692    2581 scope.go:117] "RemoveContainer" containerID="d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa"
Sep 06 19:44:35.540950 ip-10-0-135-200 kubenswrapper[2581]: E0906 19:44:35.540931    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa\": container with ID starting with d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa not found: ID does not exist" containerID="d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa"
Sep 06 19:44:35.540992 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.540956    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa"} err="failed to get container status \"d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa\": rpc error: code = NotFound desc = could not find container \"d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa\": container with ID starting with d9ebea4006eee056c1c69d5b29f250ccdb51fde170ce3e596245f38e566038fa not found: ID does not exist"
Sep 06 19:44:35.545456 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.545432    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"]
Sep 06 19:44:35.549635 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:35.549613    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-476f98-openshift-default-68c9f4497d-dsgxx"]
Sep 06 19:44:36.266583 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:36.266550    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" path="/var/lib/kubelet/pods/83122217-d60d-4ef3-9762-8d24d44641d8/volumes"
Sep 06 19:44:56.921327 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:44:56.921290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:45:00.133451 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133415    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h"]
Sep 06 19:45:00.133843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133804    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy"
Sep 06 19:45:00.133843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133815    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy"
Sep 06 19:45:00.133843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133825    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy"
Sep 06 19:45:00.133843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133831    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy"
Sep 06 19:45:00.133843 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133843    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133850    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133857    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0bef290-31f5-499f-9f13-6e0ecff9f764" containerName="revoke-keys"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133862    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0bef290-31f5-499f-9f13-6e0ecff9f764" containerName="revoke-keys"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133870    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133875    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133937    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133946    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133953    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae671dd5-c6ae-4bc1-8587-1f4906ec3f42" containerName="istio-proxy"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133959    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0bef290-31f5-499f-9f13-6e0ecff9f764" containerName="revoke-keys"
Sep 06 19:45:00.134024 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.133965    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="83122217-d60d-4ef3-9762-8d24d44641d8" containerName="istio-proxy"
Sep 06 19:45:00.137286 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.137270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h"
Sep 06 19:45:00.139226 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.139195    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpc8j\""
Sep 06 19:45:00.142828 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.142802    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h"]
Sep 06 19:45:00.150896 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.150869    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h"
Sep 06 19:45:00.424661 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.424632    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h"]
Sep 06 19:45:00.426785 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:45:00.426749    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ce7055_5c74_4374_a699_03b25b1c3f16.slice/crio-119f83665b39565249b7db29946bb10b1c458aa37aa843246a259e4546150d7b WatchSource:0}: Error finding container 119f83665b39565249b7db29946bb10b1c458aa37aa843246a259e4546150d7b: Status 404 returned error can't find the container with id 119f83665b39565249b7db29946bb10b1c458aa37aa843246a259e4546150d7b
Sep 06 19:45:00.671704 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:00.671666    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h" event={"ID":"19ce7055-5c74-4374-a699-03b25b1c3f16","Type":"ContainerStarted","Data":"119f83665b39565249b7db29946bb10b1c458aa37aa843246a259e4546150d7b"}
Sep 06 19:45:01.678905 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:01.678815    2581 generic.go:356] "Generic (PLEG): container finished" podID="19ce7055-5c74-4374-a699-03b25b1c3f16" containerID="ee6564c5e62946a555d403e3e9ba3180066add3c50c1972edd539c0ff5bd451f" exitCode=0
Sep 06 19:45:01.679302 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:01.678908    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h" event={"ID":"19ce7055-5c74-4374-a699-03b25b1c3f16","Type":"ContainerDied","Data":"ee6564c5e62946a555d403e3e9ba3180066add3c50c1972edd539c0ff5bd451f"}
Sep 06 19:45:02.820895 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:02.820869    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h"
Sep 06 19:45:03.691315 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:03.691286    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h"
Sep 06 19:45:03.691493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:03.691329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812065-swh6h" event={"ID":"19ce7055-5c74-4374-a699-03b25b1c3f16","Type":"ContainerDied","Data":"119f83665b39565249b7db29946bb10b1c458aa37aa843246a259e4546150d7b"}
Sep 06 19:45:03.691493 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:03.691372    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="119f83665b39565249b7db29946bb10b1c458aa37aa843246a259e4546150d7b"
Sep 06 19:45:12.754431 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:12.754395    2581 scope.go:117] "RemoveContainer" containerID="ec9aab454c62e94e762292ec83e710e5f59a0be8a168eb3241fe03f781c6bb45"
Sep 06 19:45:12.763379 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:12.763358    2581 scope.go:117] "RemoveContainer" containerID="501717f1401fafe685549e7d11cd02ff0dcb3240abc1a82fb3f85612da561261"
Sep 06 19:45:12.771579 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:12.771559    2581 scope.go:117] "RemoveContainer" containerID="cb3c0b1aef18577576ff5711395d9963966e1648e00fd69fcd886598d4b955fa"
Sep 06 19:45:56.022047 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:56.022011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:45:57.855418 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:57.855376    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:45:59.292286 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:45:59.292249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:46:12.429345 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:46:12.429314    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:46:12.434966 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:46:12.434941    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:46:12.438295 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:46:12.438274    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:46:12.444055 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:46:12.444032    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:48:21.187218 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:48:21.187172    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:49:07.149089 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:49:07.149031    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:49:08.848344 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:49:08.848309    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:49:12.833248 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:49:12.833210    2581 scope.go:117] "RemoveContainer" containerID="1b1c465b8be47f7d19a8d90edf83ac7178582bc1e383dfc2b5bf0df230d5c1d5"
Sep 06 19:49:12.844620 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:49:12.844588    2581 scope.go:117] "RemoveContainer" containerID="ddde78e0eabe737f4b0663e422a5882baf560824c075667a6b0a3f7e13680d3d"
Sep 06 19:49:45.174051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:49:45.174010    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:50:08.969536 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:50:08.969450    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:50:10.245470 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:50:10.245432    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:50:12.873406 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:50:12.873373    2581 scope.go:117] "RemoveContainer" containerID="db443c872713ad808d06523ffc927931c987b8f9f25547161924facc6cae89b3"
Sep 06 19:50:12.882029 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:50:12.882008    2581 scope.go:117] "RemoveContainer" containerID="01babcbaed6e9dfddef7d7c6e7a9b86a04584f6ce2f7cf214bfaeefd4508c696"
Sep 06 19:51:11.087321 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:11.087284    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nstws_716ad3b7-1459-48be-b8db-b281a92f213f/manager/0.log"
Sep 06 19:51:11.197052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:11.197016    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xhwjk_831b3d1a-6948-40b5-88a6-1fe9a71d6901/manager/0.log"
Sep 06 19:51:11.904627 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:11.904594    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d4b9b44b9-x9qtc_8bfbac4f-4890-4bdb-9f21-72e6e476d78a/copy-manifests/0.log"
Sep 06 19:51:11.910739 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:11.910707    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d4b9b44b9-x9qtc_8bfbac4f-4890-4bdb-9f21-72e6e476d78a/manager/0.log"
Sep 06 19:51:12.445318 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:12.445283    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29812050-7sbsc_aa0a1aa2-13f9-4f7f-bff5-5020ddf5e816/cleanup/0.log"
Sep 06 19:51:12.464137 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:12.464105    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:51:12.469909 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:12.469885    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:51:12.474871 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:12.474853    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:51:12.480674 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:12.480649    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4frj_da3d32c4-7c34-492c-9d9e-70f03708d30d/ovn-acl-logging/0.log"
Sep 06 19:51:12.548606 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:12.548577    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29812065-swh6h_19ce7055-5c74-4374-a699-03b25b1c3f16/cleanup/0.log"
Sep 06 19:51:12.905703 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:12.905669    2581 scope.go:117] "RemoveContainer" containerID="e1844eab4038b422dd8f3653bf950b73a8027b9a1a95ebce092d4a058aebab6a"
Sep 06 19:51:13.512051 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:13.512022    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-r79vb_4aa210f1-45f2-4319-8335-db666ca52357/manager/0.log"
Sep 06 19:51:13.618080 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:13.618040    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-5lfzh_ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1/manager/0.log"
Sep 06 19:51:14.080358 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:14.080325    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-knvzq_3f2a11d7-c9b3-4c22-9986-5ae7682a5a60/manager/0.log"
Sep 06 19:51:14.413605 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:14.413577    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h_44c88b6f-e60d-45ff-a83c-468ee68f4ba1/istio-proxy/0.log"
Sep 06 19:51:14.855883 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:14.855801    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jh9s2_d6327769-d8ea-4264-bc35-64351b5f1eb4/istio-proxy/1.log"
Sep 06 19:51:15.187995 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:15.187966    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-576f88fb4d-zrv9b_70c0a808-8fad-4a11-bcd1-4c79e8353373/router/0.log"
Sep 06 19:51:15.736104 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:15.736054    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-k9sjv_b96d2c19-5f2a-499a-ba00-2b1018db2419/main/0.log"
Sep 06 19:51:15.942195 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:15.942169    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6hpv4_021fe78f-d323-4e97-b01c-3ee387b330aa/main/0.log"
Sep 06 19:51:16.160000 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:16.159969    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9rldw_f56d030f-115b-4e32-a220-46de8655828e/main/0.log"
Sep 06 19:51:21.464435 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:21.464396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:51:21.522978 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:21.522940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log" limit=1024
Sep 06 19:51:24.245267 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:24.245237    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7djzc_5d5de6d9-7cef-4edf-8164-dbb60b3089e9/global-pull-secret-syncer/0.log"
Sep 06 19:51:24.318700 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:24.318661    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-97wm4_f578167d-aa31-47e9-841f-75928decb2e1/konnectivity-agent/0.log"
Sep 06 19:51:24.423526 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:24.423466    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-200.ec2.internal_1e2efa84dec2db25c9bb91631edcf5d2/haproxy/0.log"
Sep 06 19:51:29.281657 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:29.281624    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-r79vb_4aa210f1-45f2-4319-8335-db666ca52357/manager/0.log"
Sep 06 19:51:29.305408 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:29.305377    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-5lfzh_ff4c9d43-5ac7-4f9e-b671-bd6cb44e8bf1/manager/0.log"
Sep 06 19:51:29.469078 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:29.469040    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-knvzq_3f2a11d7-c9b3-4c22-9986-5ae7682a5a60/manager/0.log"
Sep 06 19:51:31.064962 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.064928    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-2s4kx_6903a9cf-ecd0-4d30-a595-4d32a89547c3/kube-state-metrics/0.log"
Sep 06 19:51:31.080894 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.080865    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-2s4kx_6903a9cf-ecd0-4d30-a595-4d32a89547c3/kube-rbac-proxy-main/0.log"
Sep 06 19:51:31.099304 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.099273    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-2s4kx_6903a9cf-ecd0-4d30-a595-4d32a89547c3/kube-rbac-proxy-self/0.log"
Sep 06 19:51:31.123661 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.123628    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6d998fc79-f5pnk_c2241bb1-852d-40cb-ad75-74953f81179c/metrics-server/0.log"
Sep 06 19:51:31.154595 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.154568    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-4wrd5_28288a9d-eb7e-4806-abf2-41a17b9a21ae/monitoring-plugin/0.log"
Sep 06 19:51:31.339785 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.339684    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l77bc_4060c3dc-9d38-4f92-bde1-d20bd499c9b7/node-exporter/0.log"
Sep 06 19:51:31.358559 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.358536    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l77bc_4060c3dc-9d38-4f92-bde1-d20bd499c9b7/kube-rbac-proxy/0.log"
Sep 06 19:51:31.382178 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.382153    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l77bc_4060c3dc-9d38-4f92-bde1-d20bd499c9b7/init-textfile/0.log"
Sep 06 19:51:31.516191 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.516160    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_94996cdc-40b3-4739-a8cf-7a707ff38c12/prometheus/0.log"
Sep 06 19:51:31.536998 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.536950    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_94996cdc-40b3-4739-a8cf-7a707ff38c12/config-reloader/0.log"
Sep 06 19:51:31.556958 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.556932    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_94996cdc-40b3-4739-a8cf-7a707ff38c12/thanos-sidecar/0.log"
Sep 06 19:51:31.585273 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.585202    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_94996cdc-40b3-4739-a8cf-7a707ff38c12/kube-rbac-proxy-web/0.log"
Sep 06 19:51:31.606400 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.606307    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_94996cdc-40b3-4739-a8cf-7a707ff38c12/kube-rbac-proxy/0.log"
Sep 06 19:51:31.630856 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.630827    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_94996cdc-40b3-4739-a8cf-7a707ff38c12/kube-rbac-proxy-thanos/0.log"
Sep 06 19:51:31.650867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.650842    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_94996cdc-40b3-4739-a8cf-7a707ff38c12/init-config-reloader/0.log"
Sep 06 19:51:31.721764 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.721731    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-ngvdx_244a06a3-600e-4488-9f00-fca365c23e84/prometheus-operator-admission-webhook/0.log"
Sep 06 19:51:31.835757 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.835728    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648c8d56f9-4vnhn_30a91bfe-84fd-4a0c-8bf1-07a7655a303d/thanos-query/0.log"
Sep 06 19:51:31.855749 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.855720    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648c8d56f9-4vnhn_30a91bfe-84fd-4a0c-8bf1-07a7655a303d/kube-rbac-proxy-web/0.log"
Sep 06 19:51:31.874626 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.874553    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648c8d56f9-4vnhn_30a91bfe-84fd-4a0c-8bf1-07a7655a303d/kube-rbac-proxy/0.log"
Sep 06 19:51:31.894991 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.894965    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648c8d56f9-4vnhn_30a91bfe-84fd-4a0c-8bf1-07a7655a303d/prom-label-proxy/0.log"
Sep 06 19:51:31.913351 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.913324    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648c8d56f9-4vnhn_30a91bfe-84fd-4a0c-8bf1-07a7655a303d/kube-rbac-proxy-rules/0.log"
Sep 06 19:51:31.936480 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:31.936452    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648c8d56f9-4vnhn_30a91bfe-84fd-4a0c-8bf1-07a7655a303d/kube-rbac-proxy-metrics/0.log"
Sep 06 19:51:33.907795 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:33.907685    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/1.log"
Sep 06 19:51:33.913142 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:33.913111    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ttcnn_3e3eadb1-92f7-4a9f-aa67-eab19946cbc0/console-operator/2.log"
Sep 06 19:51:34.413481 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.413445    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"]
Sep 06 19:51:34.413937 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.413918    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19ce7055-5c74-4374-a699-03b25b1c3f16" containerName="cleanup"
Sep 06 19:51:34.414035 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.413939    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="19ce7055-5c74-4374-a699-03b25b1c3f16" containerName="cleanup"
Sep 06 19:51:34.414137 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.414051    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="19ce7055-5c74-4374-a699-03b25b1c3f16" containerName="cleanup"
Sep 06 19:51:34.417351 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.417326    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.419227 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.419205    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cqltr\"/\"default-dockercfg-dr2sd\""
Sep 06 19:51:34.419327 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.419231    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cqltr\"/\"kube-root-ca.crt\""
Sep 06 19:51:34.419795 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.419781    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cqltr\"/\"openshift-service-ca.crt\""
Sep 06 19:51:34.426702 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.426677    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"]
Sep 06 19:51:34.483073 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.483024    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-sys\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.483255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.483085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-podres\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.483255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.483160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-proc\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.483255 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.483225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-lib-modules\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.483385 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.483253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9wbq\" (UniqueName: \"kubernetes.io/projected/560acf9e-fc10-48fa-9bbd-187c49dc51ab-kube-api-access-f9wbq\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.566112 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.566047    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-75xzf_23e1e106-0dcf-464c-97b0-34beec62031c/download-server/0.log"
Sep 06 19:51:34.584684 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.584645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-sys\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.584858 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.584689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-podres\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.584858 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.584744    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-proc\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.584858 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.584773    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-sys\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.584858 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.584785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-lib-modules\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.584858 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.584810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9wbq\" (UniqueName: \"kubernetes.io/projected/560acf9e-fc10-48fa-9bbd-187c49dc51ab-kube-api-access-f9wbq\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.585052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.584867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-proc\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.585052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.584884    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-podres\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.585052 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.584905    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/560acf9e-fc10-48fa-9bbd-187c49dc51ab-lib-modules\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.592813 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.592788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9wbq\" (UniqueName: \"kubernetes.io/projected/560acf9e-fc10-48fa-9bbd-187c49dc51ab-kube-api-access-f9wbq\") pod \"perf-node-gather-daemonset-98t6z\" (UID: \"560acf9e-fc10-48fa-9bbd-187c49dc51ab\") " pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:34.729406 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:34.729301    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:35.004458 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:35.004430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"]
Sep 06 19:51:35.006496 ip-10-0-135-200 kubenswrapper[2581]: W0906 19:51:35.006450    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod560acf9e_fc10_48fa_9bbd_187c49dc51ab.slice/crio-90e39baf5611d05340f0921705aff801530f583af0e4ab3920f6bdb05dd25517 WatchSource:0}: Error finding container 90e39baf5611d05340f0921705aff801530f583af0e4ab3920f6bdb05dd25517: Status 404 returned error can't find the container with id 90e39baf5611d05340f0921705aff801530f583af0e4ab3920f6bdb05dd25517
Sep 06 19:51:35.861916 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:35.861875    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z" event={"ID":"560acf9e-fc10-48fa-9bbd-187c49dc51ab","Type":"ContainerStarted","Data":"1808213504e2d153192e21ff1271e3f16ec7ff1aad2cada429e390290de6102b"}
Sep 06 19:51:35.862174 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:35.861938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z" event={"ID":"560acf9e-fc10-48fa-9bbd-187c49dc51ab","Type":"ContainerStarted","Data":"90e39baf5611d05340f0921705aff801530f583af0e4ab3920f6bdb05dd25517"}
Sep 06 19:51:35.862174 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:35.862002    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:35.875643 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:35.875592    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z" podStartSLOduration=1.875576253 podStartE2EDuration="1.875576253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:51:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:51:35.874161003 +0000 UTC m=+2424.154613798" watchObservedRunningTime="2026-09-06 19:51:35.875576253 +0000 UTC m=+2424.156029046"
Sep 06 19:51:36.400239 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:36.400203    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xc4f_cce4e1bc-30ea-4560-813d-c7c07a1f368e/dns/0.log"
Sep 06 19:51:36.417168 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:36.417141    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xc4f_cce4e1bc-30ea-4560-813d-c7c07a1f368e/kube-rbac-proxy/0.log"
Sep 06 19:51:36.529235 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:36.529200    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sf5l2_fae72b4c-8fa1-4c47-a810-601cdffc6722/dns-node-resolver/0.log"
Sep 06 19:51:37.077867 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:37.077836    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gkckz_9b36afd6-1df8-49b6-ba26-4a7e05a5dd38/node-ca/0.log"
Sep 06 19:51:37.961738 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:37.961706    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h_44c88b6f-e60d-45ff-a83c-468ee68f4ba1/istio-proxy/0.log"
Sep 06 19:51:38.324908 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:38.324818    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jh9s2_d6327769-d8ea-4264-bc35-64351b5f1eb4/istio-proxy/0.log"
Sep 06 19:51:38.335649 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:38.335589    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jh9s2_d6327769-d8ea-4264-bc35-64351b5f1eb4/istio-proxy/1.log"
Sep 06 19:51:38.414928 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:38.414898    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-576f88fb4d-zrv9b_70c0a808-8fad-4a11-bcd1-4c79e8353373/router/0.log"
Sep 06 19:51:39.238743 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:39.238716    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kvr6h_687926f7-d477-46b8-9142-c4a64f35d6e4/serve-healthcheck-canary/0.log"
Sep 06 19:51:39.783939 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:39.783908    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g77gb_e621832d-7142-4720-a723-066c1cfb5adf/kube-rbac-proxy/0.log"
Sep 06 19:51:39.801826 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:39.801798    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g77gb_e621832d-7142-4720-a723-066c1cfb5adf/exporter/0.log"
Sep 06 19:51:39.822788 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:39.822764    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g77gb_e621832d-7142-4720-a723-066c1cfb5adf/extractor/0.log"
Sep 06 19:51:41.877245 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:41.877216    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cqltr/perf-node-gather-daemonset-98t6z"
Sep 06 19:51:41.992106 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:41.992073    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nstws_716ad3b7-1459-48be-b8db-b281a92f213f/manager/0.log"
Sep 06 19:51:42.184110 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:42.184056    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xhwjk_831b3d1a-6948-40b5-88a6-1fe9a71d6901/manager/0.log"
Sep 06 19:51:42.495539 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:42.495451    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d4b9b44b9-x9qtc_8bfbac4f-4890-4bdb-9f21-72e6e476d78a/manager/0.log"
Sep 06 19:51:42.513385 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:42.513356    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d4b9b44b9-x9qtc_8bfbac4f-4890-4bdb-9f21-72e6e476d78a/copy-manifests/0.log"
Sep 06 19:51:43.971381 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:43.971348    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c4c9c58b7-g5l2j_b56c21ec-2ef5-427e-be6f-d00c51d9a71c/manager/0.log"
Sep 06 19:51:44.795971 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:44.795939    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h_44c88b6f-e60d-45ff-a83c-468ee68f4ba1/istio-proxy/0.log"
Sep 06 19:51:45.121081 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:45.121041    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jh9s2_d6327769-d8ea-4264-bc35-64351b5f1eb4/istio-proxy/0.log"
Sep 06 19:51:45.131210 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:45.131187    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jh9s2_d6327769-d8ea-4264-bc35-64351b5f1eb4/istio-proxy/1.log"
Sep 06 19:51:45.189726 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:45.189692    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-576f88fb4d-zrv9b_70c0a808-8fad-4a11-bcd1-4c79e8353373/router/0.log"
Sep 06 19:51:46.673883 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:46.673851    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-bm29h_44c88b6f-e60d-45ff-a83c-468ee68f4ba1/istio-proxy/0.log"
Sep 06 19:51:47.006243 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:47.006167    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jh9s2_d6327769-d8ea-4264-bc35-64351b5f1eb4/istio-proxy/0.log"
Sep 06 19:51:47.016285 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:47.016248    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jh9s2_d6327769-d8ea-4264-bc35-64351b5f1eb4/istio-proxy/1.log"
Sep 06 19:51:47.084999 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:47.084972    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-576f88fb4d-zrv9b_70c0a808-8fad-4a11-bcd1-4c79e8353373/router/0.log"
Sep 06 19:51:49.091557 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:49.091530    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-cz268_ca149796-d1f4-4805-a50e-3fce6118c178/migrator/0.log"
Sep 06 19:51:49.109341 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:49.109304    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-cz268_ca149796-d1f4-4805-a50e-3fce6118c178/graceful-termination/0.log"
Sep 06 19:51:49.426027 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:49.426002    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-w6lgm_e97e47da-6bc6-48f8-bf89-18da9c037980/kube-storage-version-migrator-operator/1.log"
Sep 06 19:51:49.426992 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:49.426970    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-w6lgm_e97e47da-6bc6-48f8-bf89-18da9c037980/kube-storage-version-migrator-operator/0.log"
Sep 06 19:51:50.390279 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:50.390243    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gb7wn_88928f30-fd24-43b0-9f56-3eb2192a0784/kube-multus-additional-cni-plugins/0.log"
Sep 06 19:51:50.408783 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:50.408752    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gb7wn_88928f30-fd24-43b0-9f56-3eb2192a0784/egress-router-binary-copy/0.log"
Sep 06 19:51:50.429389 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:50.429359    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gb7wn_88928f30-fd24-43b0-9f56-3eb2192a0784/cni-plugins/0.log"
Sep 06 19:51:50.449714 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:50.449690    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gb7wn_88928f30-fd24-43b0-9f56-3eb2192a0784/bond-cni-plugin/0.log"
Sep 06 19:51:50.470863 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:50.470812    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gb7wn_88928f30-fd24-43b0-9f56-3eb2192a0784/routeoverride-cni/0.log"
Sep 06 19:51:50.489022 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:50.488997    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gb7wn_88928f30-fd24-43b0-9f56-3eb2192a0784/whereabouts-cni-bincopy/0.log"
Sep 06 19:51:50.508277 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:50.508241    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gb7wn_88928f30-fd24-43b0-9f56-3eb2192a0784/whereabouts-cni/0.log"
Sep 06 19:51:50.810953 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:50.810915    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gw6vb_f9e352ae-eef3-4efd-9219-050e2aa51c2f/kube-multus/0.log"
Sep 06 19:51:51.000185 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:51.000098    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xm5z7_88608533-3551-4eaf-8716-5757cd3108c0/network-metrics-daemon/0.log"
Sep 06 19:51:51.015563 ip-10-0-135-200 kubenswrapper[2581]: I0906 19:51:51.015535    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xm5z7_88608533-3551-4eaf-8716-5757cd3108c0/kube-rbac-proxy/0.log"